{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5QNW8f3BPIoi"
   },
   "source": [
    "# Deep Learning with PyTorch Step-by-Step: A Beginner's Guide"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JQfcyrdyPIok"
   },
   "source": [
    "# Chapter 11"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AA_yhplzPIol"
   },
   "source": [
    "## Additional Setup\n",
    "\n",
    "The repository for the `allennlp` library has been archived. The library was used by `flair` to provide ELMO embeddings. This required PyTorch to be downgraded to version 1.7.1, and it prevented updating other libraries used in this chapter.\n",
    "\n",
    "At this point, the sensible solution is to upgrade `flair` and the other packages to more up-to-date versions and retire the use of ELMO embeddings. For the sake of completion, these embeddings will be replaced by flair embeddings in the code. Some code will be commented out whenever the replacement produces results that are substantially different from those produced by ELMO.\n",
    "\n",
    "Moreover, upgrading the `gensim` library required a few changes in the code since some attributes were changed from its 3.8.3 to its 4.3.1 version. These changes are highlighted in the code.\n",
    "\n",
    "**IMPORTANT**: due to the major version upgrade, the output of several cells will be quite different than those shown in the book. The new Gensim version produces slightly different vocabularies, so you should expect small changes in the number of words in the vocabulary, for example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "C8BhclloPIom"
   },
   "outputs": [],
   "source": [
    "# # UPDATED\n",
    "# ###########################################################\n",
    "# !pip install gensim==4.3.1\n",
    "# # The library has been archived and won't be used anymore\n",
    "# # # !pip install allennlp==0.9.0\n",
    "# !pip install flair==0.12.2\n",
    "# !pip install torchvision==0.15.1\n",
    "# # # HuggingFace\n",
    "# !pip install transformers==4.32.0\n",
    "# !pip install datasets==2.14.4\n",
    "# ###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w5okKRDWPIon",
    "outputId": "0c1e9290-21f7-4d4d-fd8e-27ee4141aa49"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading files from GitHub repo to Colab...\n",
      "Finished!\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    import google.colab\n",
    "    import requests\n",
    "    url = 'https://raw.githubusercontent.com/dvgodoy/PyTorchStepByStep/master/config.py'\n",
    "    r = requests.get(url, allow_redirects=True)\n",
    "    open('config.py', 'wb').write(r.content)\n",
    "except ModuleNotFoundError:\n",
    "    pass\n",
    "\n",
    "from config import *\n",
    "config_chapter11()\n",
    "# This is needed to render the plots in this chapter\n",
    "from plots.chapter11 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "MMmcGqLxPIoo"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import errno\n",
    "import requests\n",
    "import numpy as np\n",
    "from copy import deepcopy\n",
    "from operator import itemgetter\n",
    "\n",
    "import torch\n",
    "import torch.optim as optim\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import DataLoader, TensorDataset, Dataset\n",
    "\n",
    "from data_generation.nlp import ALICE_URL, WIZARD_URL, download_text\n",
    "from stepbystep.v4 import StepByStep\n",
    "# These are the classes we built in Chapter 10\n",
    "from seq2seq import *\n",
    "\n",
    "import nltk\n",
    "from nltk.tokenize import sent_tokenize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "fU1AMangTB1o"
   },
   "outputs": [],
   "source": [
    "import gensim\n",
    "from gensim import corpora, downloader\n",
    "from gensim.parsing.preprocessing import *\n",
    "from gensim.utils import simple_preprocess\n",
    "from gensim.models import Word2Vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "L6HR2OpPTtEJ"
   },
   "outputs": [],
   "source": [
    "from flair.data import Sentence\n",
    "#from flair.embeddings import ELMoEmbeddings, WordEmbeddings, \\\n",
    "#    TransformerWordEmbeddings, TransformerDocumentEmbeddings\n",
    "from flair.embeddings import WordEmbeddings, \\\n",
    "    TransformerWordEmbeddings, TransformerDocumentEmbeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "sfXyVryWT2mM"
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset, Split\n",
    "from transformers import (\n",
    "    DataCollatorForLanguageModeling,\n",
    "    BertModel, BertTokenizer, BertForSequenceClassification,\n",
    "    DistilBertModel, DistilBertTokenizer,\n",
    "    DistilBertForSequenceClassification,\n",
    "    AutoModelForSequenceClassification,\n",
    "    AutoModel, AutoTokenizer, AutoModelForCausalLM,\n",
    "    Trainer, TrainingArguments, pipeline, TextClassificationPipeline\n",
    ")\n",
    "from transformers.pipelines import SUPPORTED_TASKS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aOMOKCI3PIop"
   },
   "source": [
    "# Down the Yellow Brick Rabbit Hole"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RG2bSGFtPIop"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/alice_dorothy.png?raw=1)\n",
    "\n",
    "*Left: \"Alice and the Baby Pig\" illustration by John Tenniel's, from \"Alice's Adventure's in Wonderland\" (1865).*\n",
    "\n",
    "*Right: \"Dorothy meets the Cowardly Lion\" illustration by W.W. Denslow, from \"The Wonderful Wizard of Oz\" (1900)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zqGeN0zzPIoq"
   },
   "source": [
    "# Building a Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "YIiNrb9uPIor"
   },
   "outputs": [],
   "source": [
    "localfolder = 'texts'\n",
    "download_text(ALICE_URL, localfolder)\n",
    "download_text(WIZARD_URL, localfolder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "4IceFMuyPIor"
   },
   "outputs": [],
   "source": [
    "with open(os.path.join(localfolder, 'alice28-1476.txt'), 'r') as f:\n",
    "    alice = ''.join(f.readlines()[104:3704])\n",
    "\n",
    "with open(os.path.join(localfolder, 'wizoz10-1740.txt'), 'r') as f:\n",
    "    wizard = ''.join(f.readlines()[310:5100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Hpw53y4yPIos",
    "outputId": "9ec55a3a-aeb0-4c10-8d05-fc1dd7402e2d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                ALICE'S ADVENTURES IN WONDERLAND\n",
      "\n",
      "                          Lewis Carroll\n",
      "\n",
      "               THE MILLENNIUM FULCRUM EDITION 2.8\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "                            CHAPTER I\n",
      "\n",
      "                      Down the Rabbit-Hole\n",
      "\n",
      "\n",
      "  Alice was beginning to get very tired of sitting by her sister\n",
      "on the bank, and of having nothing to do:  once or twice she had\n",
      "peeped into the book her sister was reading, but it had no\n",
      "pictures or conversations in it, `and what is the use of a book,'\n",
      "thought Alice `w\n",
      "\n",
      "\n",
      "                    THE WONDERFUL WIZARD OF OZ\n",
      "\n",
      "\n",
      "                          1.  The Cyclone\n",
      "\n",
      "\n",
      "    Dorothy lived in the midst of the great Kansas prairies, with\n",
      "Uncle Henry, who was a farmer, and Aunt Em, who was the farmer's\n",
      "wife.  Their house was small, for the lumber to build it had to be\n",
      "carried by wagon many miles.  There were four walls, a floor and a\n",
      "roof, which made one room; and this room contained a rusty looking\n",
      "cookstove, a cupboard for the dishes, a table, three or four\n",
      "chairs, and th\n"
     ]
    }
   ],
   "source": [
    "print(alice[:500])\n",
    "print('\\n')\n",
    "print(wizard[:500])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "JNdQPUzqPIos"
   },
   "outputs": [],
   "source": [
    "text_cfg = \"\"\"fname,start,end\n",
    "alice28-1476.txt,104,3704\n",
    "wizoz10-1740.txt,310,5100\"\"\"\n",
    "bytes_written = open(os.path.join(localfolder, 'lines.cfg'), 'w').write(text_cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cd4tQSexPIot"
   },
   "source": [
    "## Sentence Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ms_jPDTWPIot",
    "outputId": "b64afbbe-9c24-4715-cc29-14fe4184979d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"I'm\", 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = \"I'm following the white rabbit\"\n",
    "tokens = sentence.split(' ')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UxbboLMMPIou",
    "outputId": "9021e58c-35e2-407c-afd8-c2a1f8bdb076"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(1612, 2240)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "from nltk.tokenize import sent_tokenize\n",
    "\n",
    "nltk.download('punkt')\n",
    "corpus_alice = sent_tokenize(alice)\n",
    "corpus_wizard = sent_tokenize(wizard)\n",
    "len(corpus_alice), len(corpus_wizard)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "IQ0TkRtRPIou",
    "outputId": "ee34b76d-4c2f-4b4b-cfa1-41ab480e1fcf"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'There was nothing so VERY remarkable in that; nor did Alice\\nthink it so VERY much out of the way to hear the Rabbit say to\\nitself, `Oh dear!'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_alice[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "gFOXRwwLPIov",
    "outputId": "ea686ce8-4815-496f-aa50-6b9e0339fc3f"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'\"There\\'s a cyclone coming, Em,\" he called to his wife.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corpus_wizard[30]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "8TX0L1NqPIov"
   },
   "outputs": [],
   "source": [
    "def sentence_tokenize(source, quote_char='\\\\', sep_char=',',\n",
    "                      include_header=True, include_source=True,\n",
    "                      extensions=('txt'), **kwargs):\n",
    "    nltk.download('punkt')\n",
    "    # If source is a folder, goes through all files inside it\n",
    "    # that match the desired extensions ('txt' by default)\n",
    "    if os.path.isdir(source):\n",
    "        filenames = [f for f in os.listdir(source)\n",
    "                     if os.path.isfile(os.path.join(source, f)) and\n",
    "                        os.path.splitext(f)[1][1:] in extensions]\n",
    "    elif isinstance(source, str):\n",
    "        filenames = [source]\n",
    "\n",
    "    # If there is a configuration file, builds a dictionary with\n",
    "    # the corresponding start and end lines of each text file\n",
    "    config_file = os.path.join(source, 'lines.cfg')\n",
    "    config = {}\n",
    "    if os.path.exists(config_file):\n",
    "        with open(config_file, 'r') as f:\n",
    "            rows = f.readlines()\n",
    "\n",
    "        for r in rows[1:]:\n",
    "            fname, start, end = r.strip().split(',')\n",
    "            config.update({fname: (int(start), int(end))})\n",
    "\n",
    "    new_fnames = []\n",
    "    # For each file of text\n",
    "    for fname in filenames:\n",
    "        # If there's a start and end line for that file, use it\n",
    "        try:\n",
    "            start, end = config[fname]\n",
    "        except KeyError:\n",
    "            start = None\n",
    "            end = None\n",
    "\n",
    "        # Opens the file, slices the configures lines (if any)\n",
    "        # cleans line breaks and uses the sentence tokenizer\n",
    "        with open(os.path.join(source, fname), 'r') as f:\n",
    "            contents = (''.join(f.readlines()[slice(start, end, None)])\n",
    "                        .replace('\\n', ' ').replace('\\r', ''))\n",
    "        corpus = sent_tokenize(contents, **kwargs)\n",
    "\n",
    "        # Builds a CSV file containing tokenized sentences\n",
    "        base = os.path.splitext(fname)[0]\n",
    "        new_fname = f'{base}.sent.csv'\n",
    "        new_fname = os.path.join(source, new_fname)\n",
    "        with open(new_fname, 'w') as f:\n",
    "            # Header of the file\n",
    "            if include_header:\n",
    "                if include_source:\n",
    "                    f.write('sentence,source\\n')\n",
    "                else:\n",
    "                    f.write('sentence\\n')\n",
    "            # Writes one line for each sentence\n",
    "            for sentence in corpus:\n",
    "                if include_source:\n",
    "                    f.write(f'{quote_char}{sentence}{quote_char}{sep_char}{fname}\\n')\n",
    "                else:\n",
    "                    f.write(f'{quote_char}{sentence}{quote_char}\\n')\n",
    "        new_fnames.append(new_fname)\n",
    "\n",
    "    # Returns list of the newly generated CSV files\n",
    "    return sorted(new_fnames)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "duMi-DzQPIov",
    "outputId": "fe2d40b8-fcdf-4228-d4d2-fe94ff0d3b0c"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['texts/alice28-1476.sent.csv', 'texts/wizoz10-1740.sent.csv']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_fnames = sentence_tokenize(localfolder)\n",
    "new_fnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "tvQ5ikWQPIow"
   },
   "outputs": [],
   "source": [
    "# import spacy\n",
    "# nlp = spacy.blank(\"en\")\n",
    "# nlp.add_pipe(nlp.create_pipe(\"sentencizer\"))\n",
    "\n",
    "# sentences = []\n",
    "# for doc in nlp.pipe(corpus_alice):\n",
    "#     sentences.extend(sent.text for sent in doc.sents)\n",
    "\n",
    "# len(sentences), sentences[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TNMT7-ClPIow"
   },
   "source": [
    "## HuggingFace's Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s5T31ZJtPIow"
   },
   "source": [
    "## Loading a Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 113,
     "referenced_widgets": [
      "4e0e6c72de6b4f01806f07631c00a8e7",
      "b63f77ac851147bb8a514d5737135f9a",
      "a000d3139bcc44babd2409da447eab16",
      "8a2aa09b29a243c2a8dca4974d3cbc7b",
      "88acd9c1e68642a8b58f9c6d77647d08",
      "2973d5f945a74243a90d44c30dd99d6c",
      "17f9f24135664f2185908a9a686fe7b8",
      "74690264747443baada58da696384387",
      "1fe2af3e027142309946a9cc019e5c13",
      "b3e74b9a6bd64af2905732e61991b1ec",
      "df0d7229eeca4861937ea11dcfb855bf",
      "dc64bd6c586c44649346778761ee47dc",
      "d2d26ca83c384e8cb8cf9996e1202c76",
      "e90b1dcef213438a91676618c66d5f85",
      "469cb766551f40f992aa5a1522f06796",
      "7d3433407dec4976b47892803044e899",
      "4629c0d2a4ca42d7b347627832900bae",
      "49fbb0bb89e1487fbdcd78e52f9a1d00",
      "bafc4fafea2344f4a7d2def7da6153d2",
      "fd192c24c0e44da0b3cf39fb1fd60b5f",
      "bcd028bfbdd44ffbb46c6e140086d03b",
      "24b4642b637b40acb9e5787c71989d1a",
      "0a85cc9e58ca491cb6ad286fc5afbcbd",
      "e74ffc2c8ab645b99b823dfcd834033e",
      "861d8b0a49d044bba3beff95743a3c47",
      "225b7a1272ad497680eead78e2fb21ba",
      "ba91e70666b84e95b4ca4e6e6e1503a5",
      "f56178de4e344ada9c275e63262ccc77",
      "8a72cf42f4e3447987928280478bbcb3",
      "61d5d21f585a4c4a8a4e0661ebab0978",
      "9c51062634f544a5afcf04f59dbc9131",
      "0591bc04e579467c9ee57bdef9210df5",
      "4c76509902e24df2aa811d754d607817"
     ]
    },
    "id": "6mmR-5ItPIox",
    "outputId": "6345bd58-fe4c-47ed-c410-56bfddfd93ff"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4e0e6c72de6b4f01806f07631c00a8e7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dc64bd6c586c44649346778761ee47dc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0a85cc9e58ca491cb6ad286fc5afbcbd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from datasets import load_dataset, Split\n",
    "\n",
    "dataset = load_dataset(path='csv', data_files=new_fnames, quotechar='\\\\', split=Split.TRAIN)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RHikb18aPIox"
   },
   "source": [
    "### Attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DvVIqWAiPIox",
    "outputId": "6a626e75-eed6-4bd8-bb9f-478ef961e29e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'sentence': Value(dtype='string', id=None),\n",
       "  'source': Value(dtype='string', id=None)},\n",
       " 2,\n",
       " (3852, 2))"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.features, dataset.num_columns, dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MtH_c0gVPIoy",
    "outputId": "1fe778ea-9428-45a0-e1bd-096ca61838db"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sentence': 'There was nothing so VERY remarkable in that; nor did Alice think it so VERY much out of the way to hear the Rabbit say to itself, `Oh dear!',\n",
       " 'source': 'alice28-1476.txt'}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xDOC9Hh4PIoy",
    "outputId": "dff8e93f-65b0-433a-8b7e-17034ac57ed9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alice28-1476.txt', 'alice28-1476.txt', 'alice28-1476.txt']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset['source'][:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EM-mEXGrPIoy"
   },
   "source": [
    "### Methods"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QprAteUMPIoy",
    "outputId": "1bb4ca60-3808-40f7-dff9-4daa784383ec"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alice28-1476.txt', 'wizoz10-1740.txt']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset.unique('source')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "9dbde3e27b4c4440b7a6de971c838ddc",
      "ff1933d7656e41cfa276d74b0d458a5d",
      "4d9eb2d890ea430cb0f6cfb03c0913ec",
      "93fa3f168798430280f1ff68660349a4",
      "63a4322a79ce4ed9ade9951a68184c79",
      "8acc01c4edd249c5bdf36333e9e24e57",
      "3d706598c13a404c882ab11ea40ed2ba",
      "1e3440f0270448baaa0b19928392e7a6",
      "1bcb447d7e7c4325bb5e590a9ad2396a",
      "466c81afbe03406280d6edd92ec0f5aa",
      "68423cc470b84d56b509e43e7d0622fc"
     ]
    },
    "id": "9XbmJVItPIoz",
    "outputId": "7d818306-08ad-4105-8551-c2f8fb03c565"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9dbde3e27b4c4440b7a6de971c838ddc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3852 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def is_alice_label(row):\n",
    "    is_alice = int(row['source'] == 'alice28-1476.txt')\n",
    "    return {'labels': is_alice}\n",
    "\n",
    "dataset = dataset.map(is_alice_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ei4rMiW0PIoz",
    "outputId": "dcf485ed-a782-4b31-8366-fc9565f70869"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sentence': 'There was nothing so VERY remarkable in that; nor did Alice think it so VERY much out of the way to hear the Rabbit say to itself, `Oh dear!',\n",
       " 'source': 'alice28-1476.txt',\n",
       " 'labels': 1}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "zL8_4qImPIo0"
   },
   "outputs": [],
   "source": [
    "shuffled_dataset = dataset.shuffle(seed=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "svODbO3NPIo0",
    "outputId": "f394dedd-992e-44ca-dc87-a3547005b8ae"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['sentence', 'source', 'labels'],\n",
       "        num_rows: 3081\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['sentence', 'source', 'labels'],\n",
       "        num_rows: 771\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "split_dataset = shuffled_dataset.train_test_split(test_size=0.2)\n",
    "split_dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "a-vhKAPsPIo0"
   },
   "outputs": [],
   "source": [
    "train_dataset, test_dataset = split_dataset['train'], split_dataset['test']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jnTeb1APPIo0"
   },
   "source": [
    "## Data Augmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "id": "v18lfIROPIo1"
   },
   "outputs": [],
   "source": [
    "# !pip install textattack\n",
    "# from textattack.augmentation import EmbeddingAugmenter\n",
    "# augmenter = EmbeddingAugmenter()\n",
    "# feynman = 'What I cannot create, I do not understand.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "oKSTr__FPIo1"
   },
   "outputs": [],
   "source": [
    "# for i in range(5):\n",
    "#     print(augmenter.augment(feynman))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1Apyc3vyPIo1"
   },
   "source": [
    "# Word Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "H6C3Qto1PIo1",
    "outputId": "fe3fc854-036c-4f58-81ab-0f0256927887"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"I'm\", 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = \"I'm following the white rabbit\"\n",
    "tokens = sentence.split(' ')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gMiBbirCPIo2",
    "outputId": "cf0a077b-3340-4d19-9330-8dc890fe3d27"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['follow', 'white', 'rabbit']"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim.parsing.preprocessing import *\n",
    "\n",
    "preprocess_string(sentence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "g9I0NFCcPIo2",
    "outputId": "273405c4-d724-41f1-ab8e-b292bded7699"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['i', 'm', 'following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filters = [lambda x: x.lower(), strip_tags, strip_punctuation, strip_multiple_whitespaces, strip_numeric]\n",
    "preprocess_string(sentence, filters=filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "I6zIZlDPPIo2",
    "outputId": "b4df3fd6-7777-43d5-c5b6-300f5aa09900"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['following', 'the', 'white', 'rabbit']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim.utils import simple_preprocess\n",
    "\n",
    "tokens = simple_preprocess(sentence)\n",
    "tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S41E3OspPIo3"
   },
   "source": [
    "## Vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Nj8VJgZ-PIo3",
    "outputId": "30552a98-c7ce-41bb-fabe-55d281dc4316"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['when',\n",
       " 'he',\n",
       " 'had',\n",
       " 'finished',\n",
       " 'he',\n",
       " 'thanked',\n",
       " 'her',\n",
       " 'kindly',\n",
       " 'and',\n",
       " 'oiled',\n",
       " 'himself',\n",
       " 'thoroughly',\n",
       " 'with',\n",
       " 'his',\n",
       " 'jeweled',\n",
       " 'oil',\n",
       " 'can',\n",
       " 'to',\n",
       " 'guard',\n",
       " 'against',\n",
       " 'mishap']"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentences = train_dataset['sentence']\n",
    "tokens = [simple_preprocess(sent) for sent in sentences]\n",
    "tokens[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RmGTVyKkPIo3",
    "outputId": "9eb66df1-8278-4ed5-b326-c6ec413271f8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dictionary<3693 unique tokens: ['against', 'and', 'can', 'finished', 'guard']...>\n"
     ]
    }
   ],
   "source": [
    "from gensim import corpora\n",
    "\n",
    "dictionary = corpora.Dictionary(tokens)\n",
    "print(dictionary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TbIfKFBIPIo3",
    "outputId": "71d928b9-93a0-4e68-9494-34f97667e6ec"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3081"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.num_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fOPK8ULTPIo4",
    "outputId": "e6826db2-2bc7-4f81-df29-cb0e50b54202"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50539"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.num_pos # processed words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3cR7uPunPIo4",
    "outputId": "7683762e-9659-4aff-89b7-2be80a5eb537",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'against': 0,\n",
       " 'and': 1,\n",
       " 'can': 2,\n",
       " 'finished': 3,\n",
       " 'guard': 4,\n",
       " 'had': 5,\n",
       " 'he': 6,\n",
       " 'her': 7,\n",
       " 'himself': 8,\n",
       " 'his': 9,\n",
       " 'jeweled': 10,\n",
       " 'kindly': 11,\n",
       " 'mishap': 12,\n",
       " 'oil': 13,\n",
       " 'oiled': 14,\n",
       " 'thanked': 15,\n",
       " 'thoroughly': 16,\n",
       " 'to': 17,\n",
       " 'when': 18,\n",
       " 'with': 19,\n",
       " 'alice': 20,\n",
       " 'cried': 21,\n",
       " 'english': 22,\n",
       " 'ever': 23,\n",
       " 'for': 24,\n",
       " 'forgot': 25,\n",
       " 'good': 26,\n",
       " 'how': 27,\n",
       " 'largest': 28,\n",
       " 'like': 29,\n",
       " 'moment': 30,\n",
       " 'much': 31,\n",
       " 'now': 32,\n",
       " 'opening': 33,\n",
       " 'out': 34,\n",
       " 'quite': 35,\n",
       " 'she': 36,\n",
       " 'so': 37,\n",
       " 'speak': 38,\n",
       " 'surprised': 39,\n",
       " 'telescope': 40,\n",
       " 'that': 41,\n",
       " 'the': 42,\n",
       " 'was': 43,\n",
       " 'able': 44,\n",
       " 'am': 45,\n",
       " 'but': 46,\n",
       " 'certainly': 47,\n",
       " 'dorothy': 48,\n",
       " 'first': 49,\n",
       " 'get': 50,\n",
       " 'help': 51,\n",
       " 'home': 52,\n",
       " 'if': 53,\n",
       " 'must': 54,\n",
       " 'replied': 55,\n",
       " 'scarecrow': 56,\n",
       " 'shall': 57,\n",
       " 'go': 58,\n",
       " 'morning': 59,\n",
       " 'returned': 60,\n",
       " 'tomorrow': 61,\n",
       " 'we': 62,\n",
       " 'came': 63,\n",
       " 'down': 64,\n",
       " 'dry': 65,\n",
       " 'fall': 66,\n",
       " 'heap': 67,\n",
       " 'leaves': 68,\n",
       " 'of': 69,\n",
       " 'over': 70,\n",
       " 'sticks': 71,\n",
       " 'upon': 72,\n",
       " 'ask': 73,\n",
       " 'give': 74,\n",
       " 'going': 75,\n",
       " 'great': 76,\n",
       " 'head': 77,\n",
       " 'him': 78,\n",
       " 'is': 79,\n",
       " 'me': 80,\n",
       " 'my': 81,\n",
       " 'oz': 82,\n",
       " 'remarked': 83,\n",
       " 'some': 84,\n",
       " 'straw': 85,\n",
       " 'stuffed': 86,\n",
       " 'dance': 87,\n",
       " 'join': 88,\n",
       " 'will': 89,\n",
       " 'won': 90,\n",
       " 'you': 91,\n",
       " 'ago': 92,\n",
       " 'balloon': 93,\n",
       " 'brought': 94,\n",
       " 'built': 95,\n",
       " 'city': 96,\n",
       " 'emerald': 97,\n",
       " 'here': 98,\n",
       " 'man': 99,\n",
       " 'many': 100,\n",
       " 'old': 101,\n",
       " 'very': 102,\n",
       " 'years': 103,\n",
       " 'young': 104,\n",
       " 'crops': 105,\n",
       " 'evidently': 106,\n",
       " 'farmers': 107,\n",
       " 'large': 108,\n",
       " 'munchkins': 109,\n",
       " 'raise': 110,\n",
       " 'were': 111,\n",
       " 'find': 112,\n",
       " 'hope': 113,\n",
       " 'again': 114,\n",
       " 'be': 115,\n",
       " 'growing': 116,\n",
       " 'small': 117,\n",
       " 'about': 118,\n",
       " 'all': 119,\n",
       " 'at': 120,\n",
       " 'back': 121,\n",
       " 'been': 122,\n",
       " 'creatures': 123,\n",
       " 'dropping': 124,\n",
       " 'figure': 125,\n",
       " 'jumping': 126,\n",
       " 'land': 127,\n",
       " 'looked': 128,\n",
       " 'mad': 129,\n",
       " 'mock': 130,\n",
       " 'quietly': 131,\n",
       " 'sadly': 132,\n",
       " 'said': 133,\n",
       " 'sat': 134,\n",
       " 'suddenly': 135,\n",
       " 'things': 136,\n",
       " 'this': 137,\n",
       " 'time': 138,\n",
       " 'turtle': 139,\n",
       " 'two': 140,\n",
       " 'voice': 141,\n",
       " 'who': 142,\n",
       " 'begins': 143,\n",
       " 'course': 144,\n",
       " 'twinkling': 145,\n",
       " 'consider': 146,\n",
       " 'in': 147,\n",
       " 'jury': 148,\n",
       " 'low': 149,\n",
       " 'trembling': 150,\n",
       " 'verdict': 151,\n",
       " 'your': 152,\n",
       " 'quick': 153,\n",
       " 'any': 154,\n",
       " 'everything': 155,\n",
       " 'green': 156,\n",
       " 'looks': 157,\n",
       " 'more': 158,\n",
       " 'no': 159,\n",
       " 'other': 160,\n",
       " 'see': 161,\n",
       " 'spectacles': 162,\n",
       " 'than': 163,\n",
       " 'wear': 164,\n",
       " 'why': 165,\n",
       " 'coward': 166,\n",
       " 'girl': 167,\n",
       " 'oh': 168,\n",
       " 'too': 169,\n",
       " 'yes': 170,\n",
       " 'after': 171,\n",
       " 'answered': 172,\n",
       " 'away': 173,\n",
       " 'frighten': 174,\n",
       " 'held': 175,\n",
       " 'lest': 176,\n",
       " 'queen': 177,\n",
       " 'ran': 178,\n",
       " 'run': 179,\n",
       " 'should': 180,\n",
       " 'they': 181,\n",
       " 'tightly': 182,\n",
       " 'toto': 183,\n",
       " 'while': 184,\n",
       " 'happened': 185,\n",
       " 'happy': 186,\n",
       " 'only': 187,\n",
       " 'telling': 188,\n",
       " 'tin': 189,\n",
       " 'us': 190,\n",
       " 'woodman': 191,\n",
       " 'aunt': 192,\n",
       " 'caught': 193,\n",
       " 'follow': 194,\n",
       " 'last': 195,\n",
       " 'started': 196,\n",
       " 'did': 197,\n",
       " 'escape': 198,\n",
       " 'manage': 199,\n",
       " 'wildcat': 200,\n",
       " 'as': 201,\n",
       " 'got': 202,\n",
       " 'helped': 203,\n",
       " 'into': 204,\n",
       " 'lion': 205,\n",
       " 'put': 206,\n",
       " 'then': 207,\n",
       " 'until': 208,\n",
       " 'up': 209,\n",
       " 'arms': 210,\n",
       " 'baby': 211,\n",
       " 'creature': 212,\n",
       " 'difficulty': 213,\n",
       " 'directions': 214,\n",
       " 'fish': 215,\n",
       " 'it': 216,\n",
       " 'its': 217,\n",
       " 'just': 218,\n",
       " 'legs': 219,\n",
       " 'little': 220,\n",
       " 'queer': 221,\n",
       " 'shaped': 222,\n",
       " 'star': 223,\n",
       " 'thought': 224,\n",
       " 'above': 225,\n",
       " 'began': 226,\n",
       " 'begun': 227,\n",
       " 'bread': 228,\n",
       " 'butter': 229,\n",
       " 'getting': 230,\n",
       " 'hadn': 231,\n",
       " 'hatter': 232,\n",
       " 'majesty': 233,\n",
       " 'not': 234,\n",
       " 'or': 235,\n",
       " 'poor': 236,\n",
       " 'tea': 237,\n",
       " 'thin': 238,\n",
       " 'week': 239,\n",
       " 'what': 240,\n",
       " 'gloves': 241,\n",
       " 'hands': 242,\n",
       " 'kid': 243,\n",
       " 'on': 244,\n",
       " 'one': 245,\n",
       " 'rabbit': 246,\n",
       " 'talking': 247,\n",
       " 'white': 248,\n",
       " 'bill': 249,\n",
       " 'chorus': 250,\n",
       " 'general': 251,\n",
       " 'goes': 252,\n",
       " 'heard': 253,\n",
       " 'there': 254,\n",
       " 'thing': 255,\n",
       " 'anything': 256,\n",
       " 'brains': 257,\n",
       " 'could': 258,\n",
       " 'decided': 259,\n",
       " 'each': 260,\n",
       " 'em': 261,\n",
       " 'friends': 262,\n",
       " 'heart': 263,\n",
       " 'kansas': 264,\n",
       " 'know': 265,\n",
       " 'matter': 266,\n",
       " 'puzzled': 267,\n",
       " 'right': 268,\n",
       " 'say': 269,\n",
       " 'wanted': 270,\n",
       " 'whether': 271,\n",
       " 'which': 272,\n",
       " 'asked': 273,\n",
       " 'become': 274,\n",
       " 'big': 275,\n",
       " 'has': 276,\n",
       " 'own': 277,\n",
       " 'shaggy': 278,\n",
       " 'witch': 279,\n",
       " 'dormouse': 280,\n",
       " 'eyes': 281,\n",
       " 'impatiently': 282,\n",
       " 'myself': 283,\n",
       " 'remark': 284,\n",
       " 'shook': 285,\n",
       " 'without': 286,\n",
       " 'air': 287,\n",
       " 'awoke': 288,\n",
       " 'beautiful': 289,\n",
       " 'country': 290,\n",
       " 'day': 291,\n",
       " 'floating': 292,\n",
       " 'found': 293,\n",
       " 'night': 294,\n",
       " 'second': 295,\n",
       " 'strange': 296,\n",
       " 'through': 297,\n",
       " 'traveled': 298,\n",
       " 'are': 299,\n",
       " 'angry': 300,\n",
       " 'bucket': 301,\n",
       " 'dashed': 302,\n",
       " 'foot': 303,\n",
       " 'from': 304,\n",
       " 'made': 305,\n",
       " 'near': 306,\n",
       " 'picked': 307,\n",
       " 'stood': 308,\n",
       " 'water': 309,\n",
       " 'wetting': 310,\n",
       " 'come': 311,\n",
       " 'conqueror': 312,\n",
       " 'daresay': 313,\n",
       " 'doesn': 314,\n",
       " 'french': 315,\n",
       " 'mouse': 316,\n",
       " 'perhaps': 317,\n",
       " 'understand': 318,\n",
       " 'william': 319,\n",
       " 'beautifully': 320,\n",
       " 'bottle': 321,\n",
       " 'carved': 322,\n",
       " 'contents': 323,\n",
       " 'cupboard': 324,\n",
       " 'dish': 325,\n",
       " 'gold': 326,\n",
       " 'high': 327,\n",
       " 'poured': 328,\n",
       " 'reaching': 329,\n",
       " 'shelf': 330,\n",
       " 'square': 331,\n",
       " 'took': 332,\n",
       " 'went': 333,\n",
       " 'exactly': 334,\n",
       " 'piece': 335,\n",
       " 'round': 336,\n",
       " 'added': 337,\n",
       " 'miles': 338,\n",
       " 'nearly': 339,\n",
       " 'among': 340,\n",
       " 'asleep': 341,\n",
       " 'closed': 342,\n",
       " 'fast': 343,\n",
       " 'fell': 344,\n",
       " 'herself': 345,\n",
       " 'poppies': 346,\n",
       " 'spite': 347,\n",
       " 'where': 348,\n",
       " 'tame': 349,\n",
       " 'do': 350,\n",
       " 'feel': 351,\n",
       " 'cross': 352,\n",
       " 'due': 353,\n",
       " 'except': 354,\n",
       " 'order': 355,\n",
       " 'place': 356,\n",
       " 'side': 357,\n",
       " 'south': 358,\n",
       " 'unwise': 359,\n",
       " 'way': 360,\n",
       " 'would': 361,\n",
       " 'also': 362,\n",
       " 'led': 363,\n",
       " 'lodged': 364,\n",
       " 'palace': 365,\n",
       " 'part': 366,\n",
       " 'pleasant': 367,\n",
       " 'rooms': 368,\n",
       " 'them': 369,\n",
       " 'these': 370,\n",
       " 'almost': 371,\n",
       " 'far': 372,\n",
       " 'feet': 373,\n",
       " 'off': 374,\n",
       " 'seemed': 375,\n",
       " 'sight': 376,\n",
       " 'have': 377,\n",
       " 'seen': 378,\n",
       " 'mouth': 379,\n",
       " 'nose': 380,\n",
       " 'child': 381,\n",
       " 'fellow': 382,\n",
       " 'sing': 383,\n",
       " 'soup': 384,\n",
       " 'bite': 385,\n",
       " 'didn': 386,\n",
       " 'hit': 387,\n",
       " 'paw': 388,\n",
       " 'rubbed': 389,\n",
       " 'king': 390,\n",
       " 'gates': 391,\n",
       " 'guardian': 392,\n",
       " 'lived': 393,\n",
       " 'reached': 394,\n",
       " 'room': 395,\n",
       " 'search': 396,\n",
       " 'soldier': 397,\n",
       " 'streets': 398,\n",
       " 'whiskers': 399,\n",
       " 'wicked': 400,\n",
       " 'better': 401,\n",
       " 'fresh': 402,\n",
       " 'might': 403,\n",
       " 'new': 404,\n",
       " 'paint': 405,\n",
       " 'don': 406,\n",
       " 'duchess': 407,\n",
       " 'fact': 408,\n",
       " 'difficult': 409,\n",
       " 'falling': 410,\n",
       " 'grass': 411,\n",
       " 'hid': 412,\n",
       " 'holes': 413,\n",
       " 'muddy': 414,\n",
       " 'thick': 415,\n",
       " 'walk': 416,\n",
       " 'bright': 417,\n",
       " 'every': 418,\n",
       " 'fields': 419,\n",
       " 'flowers': 420,\n",
       " 'journey': 421,\n",
       " 'stretched': 422,\n",
       " 'because': 423,\n",
       " 'being': 424,\n",
       " 'body': 425,\n",
       " 'cannot': 426,\n",
       " 'confidentially': 427,\n",
       " 'continued': 428,\n",
       " 'hurt': 429,\n",
       " 'mind': 430,\n",
       " 'afford': 431,\n",
       " 'couldn': 432,\n",
       " 'learn': 433,\n",
       " 'before': 434,\n",
       " 'broad': 435,\n",
       " 'carried': 436,\n",
       " 'cyclone': 437,\n",
       " 'farmhouse': 438,\n",
       " 'henry': 439,\n",
       " 'prairie': 440,\n",
       " 'sitting': 441,\n",
       " 'uncle': 442,\n",
       " 'appeared': 443,\n",
       " 'hard': 444,\n",
       " 'rush': 445,\n",
       " 'safe': 446,\n",
       " 'soon': 447,\n",
       " 'wood': 448,\n",
       " 'cornfield': 449,\n",
       " 'lifted': 450,\n",
       " 'pole': 451,\n",
       " 'beast': 452,\n",
       " 'pleasanter': 453,\n",
       " 'surely': 454,\n",
       " 'wild': 455,\n",
       " 'wish': 456,\n",
       " 'castle': 457,\n",
       " 'few': 458,\n",
       " 'melted': 459,\n",
       " 'minutes': 460,\n",
       " 'well': 461,\n",
       " 'yourself': 462,\n",
       " 'another': 463,\n",
       " 'bit': 464,\n",
       " 'dark': 465,\n",
       " 'hurrying': 466,\n",
       " 'jumped': 467,\n",
       " 'long': 468,\n",
       " 'overhead': 469,\n",
       " 'passage': 470,\n",
       " 'still': 471,\n",
       " 'answer': 472,\n",
       " 'dies': 473,\n",
       " 'cow': 474,\n",
       " 'east': 475,\n",
       " 'marriage': 476,\n",
       " 'prevent': 477,\n",
       " 'promised': 478,\n",
       " 'sheep': 479,\n",
       " 'woman': 480,\n",
       " 'kept': 481,\n",
       " 'lay': 482,\n",
       " 'sleep': 483,\n",
       " 'usual': 484,\n",
       " 'watch': 485,\n",
       " 'felt': 486,\n",
       " 'relieved': 487,\n",
       " 'set': 488,\n",
       " 'trot': 489,\n",
       " 'carefully': 490,\n",
       " 'door': 491,\n",
       " 'dress': 492,\n",
       " 'key': 493,\n",
       " 'locked': 494,\n",
       " 'pocket': 495,\n",
       " 'arm': 496,\n",
       " 'court': 497,\n",
       " 'followed': 498,\n",
       " 'hare': 499,\n",
       " 'march': 500,\n",
       " 'indeed': 501,\n",
       " 'send': 502,\n",
       " 'cook': 503,\n",
       " 'shan': 504,\n",
       " 'does': 505,\n",
       " 'maiden': 506,\n",
       " 'marry': 507,\n",
       " 'munchkin': 508,\n",
       " 'purple': 509,\n",
       " 'turning': 510,\n",
       " 'even': 511,\n",
       " 'kill': 512,\n",
       " 'breakfast': 513,\n",
       " 'saving': 514,\n",
       " 'their': 515,\n",
       " 'never': 516,\n",
       " 'pig': 517,\n",
       " 'by': 518,\n",
       " 'giving': 519,\n",
       " 'guinea': 520,\n",
       " 'lizard': 521,\n",
       " 'middle': 522,\n",
       " 'pigs': 523,\n",
       " 'something': 524,\n",
       " 'anxiously': 525,\n",
       " 'an': 526,\n",
       " 'cottage': 527,\n",
       " 'changing': 528,\n",
       " 'hastily': 529,\n",
       " 'often': 530,\n",
       " 'particular': 531,\n",
       " 'size': 532,\n",
       " 'relief': 533,\n",
       " 'shore': 534,\n",
       " 'swim': 535,\n",
       " 'toward': 536,\n",
       " 'amuse': 537,\n",
       " 'chariot': 538,\n",
       " 'courtyard': 539,\n",
       " 'cowardly': 540,\n",
       " 'draw': 541,\n",
       " 'drive': 542,\n",
       " 'harness': 543,\n",
       " 'horse': 544,\n",
       " 'make': 545,\n",
       " 'sure': 546,\n",
       " 'whenever': 547,\n",
       " 'wished': 548,\n",
       " 'work': 549,\n",
       " 'tired': 550,\n",
       " 'persisted': 551,\n",
       " 'box': 552,\n",
       " 'downwards': 553,\n",
       " 'haste': 554,\n",
       " 'melancholy': 555,\n",
       " 'move': 556,\n",
       " 'saw': 557,\n",
       " 'tail': 558,\n",
       " 'unable': 559,\n",
       " 'waving': 560,\n",
       " 'begin': 561,\n",
       " 'please': 562,\n",
       " 'harm': 563,\n",
       " 'haven': 564,\n",
       " 'else': 565,\n",
       " 'nothing': 566,\n",
       " 'caterpillar': 567,\n",
       " 'grumble': 568,\n",
       " 'trouble': 569,\n",
       " 'geography': 570,\n",
       " 'however': 571,\n",
       " 'let': 572,\n",
       " 'multiplication': 573,\n",
       " 'signify': 574,\n",
       " 'table': 575,\n",
       " 'try': 576,\n",
       " 'argue': 577,\n",
       " 'dreadful': 578,\n",
       " 'muttered': 579,\n",
       " 'really': 580,\n",
       " 'begged': 581,\n",
       " 'ears': 582,\n",
       " 'hand': 583,\n",
       " 'laugh': 584,\n",
       " 'laughter': 585,\n",
       " 'merry': 586,\n",
       " 'orphan': 587,\n",
       " 'press': 588,\n",
       " 'scream': 589,\n",
       " 'startled': 590,\n",
       " 'wonder': 591,\n",
       " 'teacup': 592,\n",
       " 'cheeks': 593,\n",
       " 'clown': 594,\n",
       " 'fair': 595,\n",
       " 'joker': 596,\n",
       " 'lady': 597,\n",
       " 'mr': 598,\n",
       " 'nodding': 599,\n",
       " 'pockets': 600,\n",
       " 'puffing': 601,\n",
       " 'saucily': 602,\n",
       " 'stare': 603,\n",
       " 'basket': 604,\n",
       " 'loud': 605,\n",
       " 'people': 606,\n",
       " 'visit': 607,\n",
       " 'duck': 608,\n",
       " 'enough': 609,\n",
       " 'frog': 610,\n",
       " 'generally': 611,\n",
       " 'means': 612,\n",
       " 'worm': 613,\n",
       " 'axe': 614,\n",
       " 'gave': 615,\n",
       " 'leaned': 616,\n",
       " 'lowered': 617,\n",
       " 'satisfaction': 618,\n",
       " 'sigh': 619,\n",
       " 'tree': 620,\n",
       " 'ate': 621,\n",
       " 'moved': 622,\n",
       " 'terrible': 623,\n",
       " 'direction': 624,\n",
       " 'orders': 625,\n",
       " 'possible': 626,\n",
       " 'different': 627,\n",
       " 'blade': 628,\n",
       " 'burnished': 629,\n",
       " 'glistened': 630,\n",
       " 'others': 631,\n",
       " 'polished': 632,\n",
       " 'removed': 633,\n",
       " 'rust': 634,\n",
       " 'silver': 635,\n",
       " 'egg': 636,\n",
       " 'll': 637,\n",
       " 'next': 638,\n",
       " 'suppose': 639,\n",
       " 'tasted': 640,\n",
       " 'beginning': 641,\n",
       " 'happens': 642,\n",
       " 'interrupted': 643,\n",
       " 'changed': 644,\n",
       " 'completely': 645,\n",
       " 'fan': 646,\n",
       " 'glass': 647,\n",
       " 'guessed': 648,\n",
       " 'hall': 649,\n",
       " 'hunting': 650,\n",
       " 'looking': 651,\n",
       " 'naturedly': 652,\n",
       " 'nowhere': 653,\n",
       " 'pair': 654,\n",
       " 'pool': 655,\n",
       " 'since': 656,\n",
       " 'vanished': 657,\n",
       " 'keep': 658,\n",
       " 'temper': 659,\n",
       " 'greatly': 660,\n",
       " 'pleased': 661,\n",
       " 'everybody': 662,\n",
       " 'cover': 663,\n",
       " 'distance': 664,\n",
       " 'flames': 665,\n",
       " 'farmer': 666,\n",
       " 'midst': 667,\n",
       " 'prairies': 668,\n",
       " 'wife': 669,\n",
       " 'dear': 670,\n",
       " 'forget': 671,\n",
       " 'makes': 672,\n",
       " 're': 673,\n",
       " 'talk': 674,\n",
       " 'thinking': 675,\n",
       " 'beside': 676,\n",
       " 'once': 677,\n",
       " 'sound': 678,\n",
       " 'cleared': 679,\n",
       " 'gryphon': 680,\n",
       " 'jelly': 681,\n",
       " 'salmon': 682,\n",
       " 'seals': 683,\n",
       " 'takes': 684,\n",
       " 'turtles': 685,\n",
       " 've': 686,\n",
       " 'curious': 687,\n",
       " 'feeling': 688,\n",
       " 'glad': 689,\n",
       " 'interesting': 690,\n",
       " 'song': 691,\n",
       " 'thank': 692,\n",
       " 'whiting': 693,\n",
       " 'screamed': 694,\n",
       " 'mournful': 695,\n",
       " 'tone': 696,\n",
       " 'carry': 697,\n",
       " 'pat': 698,\n",
       " 'tell': 699,\n",
       " 'window': 700,\n",
       " 'around': 701,\n",
       " 'bell': 702,\n",
       " 'eat': 703,\n",
       " 'fastened': 704,\n",
       " 'filled': 705,\n",
       " 'kind': 706,\n",
       " 'neck': 707,\n",
       " 'ribbon': 708,\n",
       " 'cats': 709,\n",
       " 'ease': 710,\n",
       " 'edge': 711,\n",
       " 'farther': 712,\n",
       " 'four': 713,\n",
       " 'travelers': 714,\n",
       " 'trees': 715,\n",
       " 'walked': 716,\n",
       " 'anyone': 717,\n",
       " 'known': 718,\n",
       " 'magic': 719,\n",
       " 'used': 720,\n",
       " 'dismay': 721,\n",
       " 'our': 722,\n",
       " 'surprise': 723,\n",
       " 'brightly': 724,\n",
       " 'moon': 725,\n",
       " 'shone': 726,\n",
       " 'walking': 727,\n",
       " 'life': 728,\n",
       " 'hear': 729,\n",
       " 'inside': 730,\n",
       " 'making': 731,\n",
       " 'noise': 732,\n",
       " 'possibly': 733,\n",
       " 'same': 734,\n",
       " 'secondly': 735,\n",
       " 'such': 736,\n",
       " 'wizards': 737,\n",
       " 'called': 738,\n",
       " 'entered': 739,\n",
       " 'became': 740,\n",
       " 'deaf': 741,\n",
       " 'fright': 742,\n",
       " 'hour': 743,\n",
       " 'lonely': 744,\n",
       " 'loudly': 745,\n",
       " 'passed': 746,\n",
       " 'shrieked': 747,\n",
       " 'slowly': 748,\n",
       " 'wind': 749,\n",
       " 'desire': 750,\n",
       " 'earned': 751,\n",
       " 'seek': 752,\n",
       " 'authority': 753,\n",
       " 'listen': 754,\n",
       " 'person': 755,\n",
       " 'sit': 756,\n",
       " 'brightened': 757,\n",
       " 'behind': 758,\n",
       " 'open': 759,\n",
       " 'pulled': 760,\n",
       " 'screen': 761,\n",
       " 'thread': 762,\n",
       " 'mouths': 763,\n",
       " 'reason': 764,\n",
       " 'shut': 765,\n",
       " 'tails': 766,\n",
       " 'yawned': 767,\n",
       " 'evening': 768,\n",
       " 'house': 769,\n",
       " 'larger': 770,\n",
       " 'pass': 771,\n",
       " 'rather': 772,\n",
       " 'rest': 773,\n",
       " 'although': 774,\n",
       " 'current': 775,\n",
       " 'drawn': 776,\n",
       " 'push': 777,\n",
       " 'raft': 778,\n",
       " 'both': 779,\n",
       " 'light': 780,\n",
       " 'west': 781,\n",
       " 'left': 782,\n",
       " 'sorrowfully': 783,\n",
       " 'throne': 784,\n",
       " 'waiting': 785,\n",
       " 'sorry': 786,\n",
       " 'truly': 787,\n",
       " 'bowing': 788,\n",
       " 'children': 789,\n",
       " 'gardeners': 790,\n",
       " 'instantly': 791,\n",
       " 'royal': 792,\n",
       " 'shrill': 793,\n",
       " 'three': 794,\n",
       " 'crab': 795,\n",
       " 'boq': 796,\n",
       " 'shoes': 797,\n",
       " 'sorceress': 798,\n",
       " 'fun': 799,\n",
       " 'clock': 800,\n",
       " 'frightened': 801,\n",
       " 'given': 802,\n",
       " 'message': 803,\n",
       " 'nine': 804,\n",
       " 'sent': 805,\n",
       " 'wizard': 806,\n",
       " 'word': 807,\n",
       " 'drunk': 808,\n",
       " 'grow': 809,\n",
       " 'saying': 810,\n",
       " 'gray': 811,\n",
       " 'lips': 812,\n",
       " 'red': 813,\n",
       " 'sober': 814,\n",
       " 'sparkle': 815,\n",
       " 'taken': 816,\n",
       " 'mine': 817,\n",
       " 'though': 818,\n",
       " 'bowed': 819,\n",
       " 'hearts': 820,\n",
       " 'knave': 821,\n",
       " 'reply': 822,\n",
       " 'smiled': 823,\n",
       " 'command': 824,\n",
       " 'world': 825,\n",
       " 'agreed': 826,\n",
       " 'cap': 827,\n",
       " 'condition': 828,\n",
       " 'golden': 829,\n",
       " 'grandfather': 830,\n",
       " 'may': 831,\n",
       " 'monkeys': 832,\n",
       " 'owner': 833,\n",
       " 'slaves': 834,\n",
       " 'times': 835,\n",
       " 'whosoever': 836,\n",
       " 'branches': 837,\n",
       " 'immediately': 838,\n",
       " 'seized': 839,\n",
       " 'spoke': 840,\n",
       " 'tossed': 841,\n",
       " 'days': 842,\n",
       " 'wrong': 843,\n",
       " 'fender': 844,\n",
       " 'hearthrug': 845,\n",
       " 'love': 846,\n",
       " 'dodo': 847,\n",
       " 'managed': 848,\n",
       " 'winter': 849,\n",
       " 'boy': 850,\n",
       " 'dangerous': 851,\n",
       " 'dwells': 852,\n",
       " 'father': 853,\n",
       " 'nearer': 854,\n",
       " 'best': 855,\n",
       " 'done': 856,\n",
       " 'forgave': 857,\n",
       " 'promise': 858,\n",
       " 'brother': 859,\n",
       " 'clouds': 860,\n",
       " 'ready': 861,\n",
       " 'confused': 862,\n",
       " 'shouldn': 863,\n",
       " 'think': 864,\n",
       " 'guests': 865,\n",
       " 'live': 866,\n",
       " 'meantime': 867,\n",
       " 'obey': 868,\n",
       " 'slightest': 869,\n",
       " 'treated': 870,\n",
       " 'wait': 871,\n",
       " 'planning': 872,\n",
       " 'bed': 873,\n",
       " 'flesh': 874,\n",
       " 'force': 875,\n",
       " 'habit': 876,\n",
       " 'joints': 877,\n",
       " 'moving': 878,\n",
       " 'remembered': 879,\n",
       " 'working': 880,\n",
       " 'bounding': 881,\n",
       " 'growl': 882,\n",
       " 'hinges': 883,\n",
       " 'worked': 884,\n",
       " 'tut': 885,\n",
       " 'dog': 886,\n",
       " 'eagerly': 887,\n",
       " 'nice': 888,\n",
       " 'show': 889,\n",
       " 'heads': 890,\n",
       " 'boots': 891,\n",
       " 'bundle': 892,\n",
       " 'clothes': 893,\n",
       " 'hat': 894,\n",
       " 'tall': 895,\n",
       " 'threw': 896,\n",
       " 'top': 897,\n",
       " 'call': 898,\n",
       " 'field': 899,\n",
       " 'mice': 900,\n",
       " 'suggested': 901,\n",
       " 'arranged': 902,\n",
       " 'bark': 903,\n",
       " 'doubt': 904,\n",
       " 'excellent': 905,\n",
       " 'hurry': 906,\n",
       " 'idea': 907,\n",
       " 'look': 908,\n",
       " 'neatly': 909,\n",
       " 'peering': 910,\n",
       " 'plan': 911,\n",
       " 'sharp': 912,\n",
       " 'simply': 913,\n",
       " 'smallest': 914,\n",
       " 'sounded': 915,\n",
       " 'wow': 916,\n",
       " 'name': 917,\n",
       " 'tossing': 918,\n",
       " 'gone': 919,\n",
       " 'rule': 920,\n",
       " 'wise': 921,\n",
       " 'bees': 922,\n",
       " 'black': 923,\n",
       " 'coal': 924,\n",
       " 'fine': 925,\n",
       " 'gnashed': 926,\n",
       " 'hair': 927,\n",
       " 'heaps': 928,\n",
       " 'stamped': 929,\n",
       " 'teeth': 930,\n",
       " 'tore': 931,\n",
       " 'altogether': 932,\n",
       " 'candle': 933,\n",
       " 'end': 934,\n",
       " 'further': 935,\n",
       " 'nervous': 936,\n",
       " 'shrink': 937,\n",
       " 'waited': 938,\n",
       " 'dinah': 939,\n",
       " 'itself': 940,\n",
       " 'pinch': 941,\n",
       " 'singing': 942,\n",
       " 'stop': 943,\n",
       " 'twinkle': 944,\n",
       " 'common': 945,\n",
       " 'cakes': 946,\n",
       " 'floor': 947,\n",
       " 'noticed': 948,\n",
       " 'pebbles': 949,\n",
       " 'delightful': 950,\n",
       " 'ropes': 951,\n",
       " 'rose': 952,\n",
       " 'rules': 953,\n",
       " 'wisely': 954,\n",
       " 'question': 955,\n",
       " 'repeated': 956,\n",
       " 'turned': 957,\n",
       " 'carrying': 958,\n",
       " 'crimson': 959,\n",
       " 'crown': 960,\n",
       " 'cushion': 961,\n",
       " 'grand': 962,\n",
       " 'procession': 963,\n",
       " 'velvet': 964,\n",
       " 'ride': 965,\n",
       " 'earth': 966,\n",
       " 'happiest': 967,\n",
       " 'resolved': 968,\n",
       " 'allow': 969,\n",
       " 'care': 970,\n",
       " 'destroy': 971,\n",
       " 'fierce': 972,\n",
       " 'take': 973,\n",
       " 'dishes': 974,\n",
       " 'washing': 975,\n",
       " 'admit': 976,\n",
       " 'alone': 977,\n",
       " 'audience': 978,\n",
       " 'enter': 979,\n",
       " 'grant': 980,\n",
       " 'presence': 981,\n",
       " 'probably': 982,\n",
       " 'awfully': 983,\n",
       " 'balanced': 984,\n",
       " 'clever': 985,\n",
       " 'eel': 986,\n",
       " 'eye': 987,\n",
       " 'hardly': 988,\n",
       " 'steady': 989,\n",
       " 'yet': 990,\n",
       " 'youth': 991,\n",
       " 'twelfth': 992,\n",
       " 'nor': 993,\n",
       " 'wall': 994,\n",
       " 'watched': 995,\n",
       " 'opened': 996,\n",
       " 'attacked': 997,\n",
       " 'bye': 998,\n",
       " 'fighting': 999,\n",
       " ...}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.token2id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "B2R6GIb0PIo5",
    "outputId": "ab99adf7-0403-4f24-ea93-e393402b9966"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['against', 'and', 'can', 'finished', 'guard']"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab = list(dictionary.token2id.keys())\n",
    "vocab[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MaZdrWZDPIo5",
    "outputId": "31b0e7d0-395f-41c5-f022-92c5bce990b3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{18: 191,\n",
       " 6: 476,\n",
       " 5: 339,\n",
       " 3: 16,\n",
       " 15: 9,\n",
       " 7: 534,\n",
       " 11: 6,\n",
       " 1: 2013,\n",
       " 14: 5,\n",
       " 8: 26,\n",
       " 16: 4,\n",
       " 19: 365,\n",
       " 9: 243,\n",
       " 10: 2,\n",
       " 13: 16,\n",
       " 2: 117,\n",
       " 17: 1450,\n",
       " 4: 3,\n",
       " 0: 12,\n",
       " 12: 3,\n",
       " 21: 36,\n",
       " 20: 319,\n",
       " 36: 735,\n",
       " 43: 661,\n",
       " 37: 361,\n",
       " 31: 74,\n",
       " 39: 16,\n",
       " 41: 558,\n",
       " 24: 377,\n",
       " 42: 3642,\n",
       " 30: 39,\n",
       " 35: 68,\n",
       " 25: 3,\n",
       " 27: 104,\n",
       " 38: 18,\n",
       " 26: 101,\n",
       " 22: 6,\n",
       " 32: 116,\n",
       " 33: 5,\n",
       " 34: 179,\n",
       " 29: 123,\n",
       " 28: 1,\n",
       " 40: 2,\n",
       " 23: 56,\n",
       " 47: 23,\n",
       " 57: 91,\n",
       " 53: 199,\n",
       " 45: 92,\n",
       " 44: 8,\n",
       " 56: 183,\n",
       " 55: 70,\n",
       " 46: 380,\n",
       " 54: 96,\n",
       " 51: 36,\n",
       " 48: 292,\n",
       " 50: 112,\n",
       " 52: 16,\n",
       " 49: 80,\n",
       " 62: 165,\n",
       " 58: 94,\n",
       " 61: 7,\n",
       " 59: 26,\n",
       " 60: 16,\n",
       " 64: 159,\n",
       " 63: 114,\n",
       " 72: 82,\n",
       " 67: 3,\n",
       " 69: 1057,\n",
       " 71: 2,\n",
       " 65: 12,\n",
       " 68: 10,\n",
       " 66: 11,\n",
       " 70: 105,\n",
       " 75: 45,\n",
       " 76: 145,\n",
       " 82: 135,\n",
       " 73: 32,\n",
       " 78: 183,\n",
       " 74: 50,\n",
       " 80: 215,\n",
       " 84: 78,\n",
       " 83: 21,\n",
       " 81: 205,\n",
       " 77: 121,\n",
       " 79: 302,\n",
       " 86: 27,\n",
       " 85: 29,\n",
       " 89: 167,\n",
       " 91: 748,\n",
       " 90: 31,\n",
       " 88: 10,\n",
       " 87: 11,\n",
       " 97: 46,\n",
       " 96: 69,\n",
       " 95: 9,\n",
       " 100: 41,\n",
       " 103: 10,\n",
       " 92: 5,\n",
       " 104: 11,\n",
       " 99: 65,\n",
       " 93: 21,\n",
       " 94: 13,\n",
       " 98: 68,\n",
       " 102: 186,\n",
       " 101: 46,\n",
       " 106: 3,\n",
       " 109: 15,\n",
       " 111: 232,\n",
       " 107: 1,\n",
       " 110: 2,\n",
       " 108: 31,\n",
       " 105: 2,\n",
       " 113: 9,\n",
       " 112: 44,\n",
       " 115: 264,\n",
       " 116: 11,\n",
       " 117: 27,\n",
       " 114: 134,\n",
       " 121: 108,\n",
       " 127: 33,\n",
       " 119: 331,\n",
       " 125: 7,\n",
       " 133: 647,\n",
       " 130: 43,\n",
       " 139: 45,\n",
       " 135: 14,\n",
       " 124: 1,\n",
       " 141: 71,\n",
       " 140: 48,\n",
       " 123: 12,\n",
       " 142: 145,\n",
       " 122: 71,\n",
       " 126: 4,\n",
       " 118: 112,\n",
       " 129: 15,\n",
       " 136: 37,\n",
       " 137: 218,\n",
       " 138: 87,\n",
       " 134: 37,\n",
       " 132: 10,\n",
       " 131: 6,\n",
       " 128: 80,\n",
       " 120: 364,\n",
       " 144: 40,\n",
       " 145: 4,\n",
       " 143: 5,\n",
       " 146: 6,\n",
       " 152: 111,\n",
       " 151: 4,\n",
       " 148: 15,\n",
       " 147: 681,\n",
       " 149: 26,\n",
       " 150: 6,\n",
       " 153: 7,\n",
       " 159: 156,\n",
       " 158: 70,\n",
       " 163: 58,\n",
       " 154: 71,\n",
       " 160: 72,\n",
       " 164: 7,\n",
       " 156: 85,\n",
       " 162: 9,\n",
       " 165: 68,\n",
       " 155: 31,\n",
       " 161: 137,\n",
       " 157: 5,\n",
       " 168: 68,\n",
       " 170: 26,\n",
       " 167: 77,\n",
       " 166: 11,\n",
       " 169: 46,\n",
       " 181: 436,\n",
       " 172: 65,\n",
       " 173: 75,\n",
       " 177: 81,\n",
       " 178: 29,\n",
       " 184: 48,\n",
       " 175: 10,\n",
       " 183: 70,\n",
       " 182: 3,\n",
       " 176: 2,\n",
       " 180: 66,\n",
       " 179: 11,\n",
       " 171: 100,\n",
       " 174: 4,\n",
       " 187: 79,\n",
       " 190: 69,\n",
       " 189: 106,\n",
       " 191: 147,\n",
       " 188: 6,\n",
       " 185: 15,\n",
       " 186: 12,\n",
       " 193: 17,\n",
       " 195: 50,\n",
       " 196: 18,\n",
       " 194: 6,\n",
       " 192: 23,\n",
       " 197: 108,\n",
       " 199: 6,\n",
       " 198: 6,\n",
       " 200: 4,\n",
       " 207: 163,\n",
       " 205: 137,\n",
       " 202: 48,\n",
       " 209: 161,\n",
       " 203: 9,\n",
       " 206: 55,\n",
       " 204: 123,\n",
       " 208: 50,\n",
       " 201: 469,\n",
       " 211: 15,\n",
       " 213: 4,\n",
       " 216: 797,\n",
       " 221: 19,\n",
       " 222: 4,\n",
       " 220: 203,\n",
       " 212: 9,\n",
       " 217: 77,\n",
       " 210: 35,\n",
       " 219: 20,\n",
       " 214: 6,\n",
       " 218: 80,\n",
       " 223: 1,\n",
       " 215: 6,\n",
       " 224: 100,\n",
       " 236: 29,\n",
       " 233: 12,\n",
       " 232: 50,\n",
       " 226: 73,\n",
       " 231: 5,\n",
       " 227: 8,\n",
       " 237: 18,\n",
       " 234: 309,\n",
       " 225: 5,\n",
       " 239: 3,\n",
       " 235: 90,\n",
       " 240: 183,\n",
       " 228: 15,\n",
       " 229: 10,\n",
       " 230: 21,\n",
       " 238: 3,\n",
       " 242: 23,\n",
       " 244: 268,\n",
       " 245: 177,\n",
       " 246: 41,\n",
       " 248: 33,\n",
       " 243: 4,\n",
       " 241: 7,\n",
       " 247: 12,\n",
       " 255: 61,\n",
       " 253: 40,\n",
       " 251: 3,\n",
       " 250: 7,\n",
       " 254: 173,\n",
       " 252: 8,\n",
       " 249: 15,\n",
       " 269: 57,\n",
       " 256: 29,\n",
       " 267: 10,\n",
       " 265: 112,\n",
       " 272: 93,\n",
       " 262: 29,\n",
       " 268: 42,\n",
       " 259: 7,\n",
       " 258: 153,\n",
       " 264: 39,\n",
       " 261: 20,\n",
       " 266: 11,\n",
       " 271: 16,\n",
       " 257: 46,\n",
       " 263: 54,\n",
       " 260: 19,\n",
       " 270: 9,\n",
       " 279: 99,\n",
       " 275: 35,\n",
       " 278: 4,\n",
       " 273: 104,\n",
       " 276: 35,\n",
       " 277: 23,\n",
       " 274: 8,\n",
       " 280: 32,\n",
       " 285: 13,\n",
       " 282: 5,\n",
       " 286: 33,\n",
       " 281: 68,\n",
       " 284: 8,\n",
       " 283: 17,\n",
       " 291: 59,\n",
       " 294: 24,\n",
       " 298: 2,\n",
       " 297: 50,\n",
       " 287: 40,\n",
       " 295: 7,\n",
       " 288: 2,\n",
       " 293: 61,\n",
       " 292: 1,\n",
       " 296: 21,\n",
       " 289: 42,\n",
       " 290: 53,\n",
       " 299: 160,\n",
       " 305: 87,\n",
       " 300: 14,\n",
       " 307: 8,\n",
       " 301: 3,\n",
       " 309: 24,\n",
       " 308: 33,\n",
       " 306: 29,\n",
       " 302: 5,\n",
       " 310: 1,\n",
       " 304: 127,\n",
       " 303: 15,\n",
       " 317: 22,\n",
       " 314: 17,\n",
       " 318: 11,\n",
       " 313: 1,\n",
       " 315: 4,\n",
       " 316: 41,\n",
       " 311: 95,\n",
       " 319: 6,\n",
       " 312: 2,\n",
       " 333: 98,\n",
       " 324: 7,\n",
       " 329: 2,\n",
       " 327: 15,\n",
       " 330: 4,\n",
       " 332: 44,\n",
       " 331: 2,\n",
       " 321: 7,\n",
       " 323: 1,\n",
       " 328: 3,\n",
       " 326: 6,\n",
       " 325: 6,\n",
       " 320: 5,\n",
       " 322: 2,\n",
       " 334: 13,\n",
       " 335: 7,\n",
       " 336: 34,\n",
       " 339: 19,\n",
       " 338: 6,\n",
       " 337: 26,\n",
       " 342: 5,\n",
       " 347: 6,\n",
       " 345: 84,\n",
       " 348: 76,\n",
       " 344: 26,\n",
       " 340: 20,\n",
       " 346: 7,\n",
       " 343: 22,\n",
       " 341: 18,\n",
       " 349: 1,\n",
       " 350: 169,\n",
       " 351: 18,\n",
       " 352: 16,\n",
       " 356: 24,\n",
       " 355: 5,\n",
       " 357: 33,\n",
       " 361: 169,\n",
       " 359: 1,\n",
       " 360: 95,\n",
       " 354: 11,\n",
       " 353: 1,\n",
       " 358: 16,\n",
       " 370: 39,\n",
       " 362: 25,\n",
       " 363: 13,\n",
       " 368: 5,\n",
       " 369: 256,\n",
       " 364: 1,\n",
       " 367: 4,\n",
       " 366: 13,\n",
       " 365: 20,\n",
       " 373: 35,\n",
       " 375: 42,\n",
       " 371: 11,\n",
       " 376: 16,\n",
       " 372: 27,\n",
       " 374: 87,\n",
       " 377: 189,\n",
       " 378: 28,\n",
       " 380: 16,\n",
       " 379: 22,\n",
       " 381: 25,\n",
       " 383: 6,\n",
       " 384: 16,\n",
       " 382: 6,\n",
       " 386: 15,\n",
       " 385: 5,\n",
       " 389: 2,\n",
       " 388: 4,\n",
       " 387: 3,\n",
       " 390: 70,\n",
       " 396: 4,\n",
       " 400: 56,\n",
       " 397: 19,\n",
       " 399: 11,\n",
       " 398: 4,\n",
       " 394: 16,\n",
       " 395: 60,\n",
       " 392: 11,\n",
       " 391: 11,\n",
       " 393: 16,\n",
       " 402: 4,\n",
       " 404: 18,\n",
       " 405: 5,\n",
       " 403: 33,\n",
       " 401: 20,\n",
       " 406: 81,\n",
       " 407: 34,\n",
       " 408: 7,\n",
       " 409: 3,\n",
       " 416: 16,\n",
       " 410: 5,\n",
       " 414: 1,\n",
       " 413: 3,\n",
       " 411: 19,\n",
       " 415: 11,\n",
       " 412: 3,\n",
       " 421: 23,\n",
       " 419: 7,\n",
       " 417: 17,\n",
       " 420: 18,\n",
       " 422: 6,\n",
       " 418: 31,\n",
       " 428: 18,\n",
       " 427: 1,\n",
       " 430: 26,\n",
       " 425: 16,\n",
       " 424: 33,\n",
       " 423: 27,\n",
       " 426: 30,\n",
       " 429: 23,\n",
       " 432: 10,\n",
       " 431: 2,\n",
       " 433: 5,\n",
       " 441: 17,\n",
       " 435: 3,\n",
       " 440: 2,\n",
       " 434: 81,\n",
       " 438: 2,\n",
       " 442: 14,\n",
       " 439: 12,\n",
       " 437: 8,\n",
       " 436: 22,\n",
       " 445: 2,\n",
       " 443: 9,\n",
       " 444: 25,\n",
       " 447: 57,\n",
       " 446: 3,\n",
       " 448: 13,\n",
       " 450: 5,\n",
       " 451: 12,\n",
       " 449: 7,\n",
       " 454: 8,\n",
       " 455: 9,\n",
       " 452: 18,\n",
       " 456: 41,\n",
       " 453: 2,\n",
       " 461: 111,\n",
       " 458: 24,\n",
       " 460: 14,\n",
       " 459: 9,\n",
       " 457: 18,\n",
       " 462: 15,\n",
       " 464: 22,\n",
       " 467: 14,\n",
       " 465: 17,\n",
       " 469: 1,\n",
       " 463: 38,\n",
       " 468: 81,\n",
       " 470: 4,\n",
       " 471: 26,\n",
       " 466: 2,\n",
       " 472: 12,\n",
       " 473: 1,\n",
       " 480: 28,\n",
       " 475: 15,\n",
       " 478: 11,\n",
       " 479: 3,\n",
       " 474: 3,\n",
       " 477: 2,\n",
       " 476: 1,\n",
       " 482: 22,\n",
       " 483: 12,\n",
       " 481: 21,\n",
       " 485: 10,\n",
       " 484: 6,\n",
       " 488: 33,\n",
       " 486: 30,\n",
       " 487: 1,\n",
       " 489: 1,\n",
       " 491: 46,\n",
       " 494: 4,\n",
       " 493: 8,\n",
       " 490: 14,\n",
       " 495: 7,\n",
       " 492: 6,\n",
       " 500: 27,\n",
       " 499: 25,\n",
       " 498: 17,\n",
       " 497: 11,\n",
       " 496: 17,\n",
       " 501: 34,\n",
       " 502: 14,\n",
       " 504: 7,\n",
       " 503: 11,\n",
       " 505: 14,\n",
       " 508: 11,\n",
       " 506: 3,\n",
       " 507: 3,\n",
       " 510: 11,\n",
       " 509: 4,\n",
       " 511: 34,\n",
       " 512: 11,\n",
       " 514: 6,\n",
       " 513: 4,\n",
       " 515: 118,\n",
       " 516: 79,\n",
       " 517: 9,\n",
       " 521: 4,\n",
       " 522: 19,\n",
       " 518: 115,\n",
       " 520: 4,\n",
       " 523: 5,\n",
       " 519: 5,\n",
       " 524: 22,\n",
       " 525: 17,\n",
       " 526: 75,\n",
       " 527: 6,\n",
       " 531: 4,\n",
       " 532: 12,\n",
       " 529: 12,\n",
       " 528: 2,\n",
       " 530: 8,\n",
       " 533: 3,\n",
       " 535: 8,\n",
       " 536: 16,\n",
       " 534: 7,\n",
       " 549: 24,\n",
       " 539: 2,\n",
       " 543: 3,\n",
       " 540: 20,\n",
       " 544: 5,\n",
       " 537: 2,\n",
       " 546: 43,\n",
       " 545: 56,\n",
       " 541: 7,\n",
       " 538: 1,\n",
       " 547: 7,\n",
       " 548: 6,\n",
       " 542: 4,\n",
       " 550: 17,\n",
       " 551: 2,\n",
       " 552: 13,\n",
       " 557: 56,\n",
       " 554: 1,\n",
       " 553: 1,\n",
       " 560: 5,\n",
       " 558: 15,\n",
       " 555: 5,\n",
       " 559: 2,\n",
       " 556: 7,\n",
       " 561: 11,\n",
       " 562: 26,\n",
       " 564: 9,\n",
       " 563: 10,\n",
       " 566: 52,\n",
       " 565: 13,\n",
       " 567: 24,\n",
       " 568: 1,\n",
       " 569: 12,\n",
       " 571: 22,\n",
       " 573: 1,\n",
       " 575: 15,\n",
       " 574: 1,\n",
       " 572: 37,\n",
       " 576: 24,\n",
       " 570: 1,\n",
       " 580: 29,\n",
       " 578: 7,\n",
       " 579: 2,\n",
       " 577: 1,\n",
       " 581: 1,\n",
       " 587: 1,\n",
       " 590: 4,\n",
       " 585: 3,\n",
       " 589: 2,\n",
       " 588: 1,\n",
       " 583: 23,\n",
       " 586: 1,\n",
       " 582: 12,\n",
       " 591: 23,\n",
       " 584: 5,\n",
       " 592: 3,\n",
       " 594: 3,\n",
       " 600: 1,\n",
       " 601: 1,\n",
       " 593: 4,\n",
       " 599: 1,\n",
       " 602: 1,\n",
       " 597: 13,\n",
       " 595: 2,\n",
       " 603: 1,\n",
       " 598: 3,\n",
       " 596: 3,\n",
       " 604: 19,\n",
       " 606: 55,\n",
       " 605: 11,\n",
       " 607: 4,\n",
       " 612: 6,\n",
       " 609: 24,\n",
       " 608: 3,\n",
       " 611: 7,\n",
       " 610: 3,\n",
       " 613: 1,\n",
       " 615: 28,\n",
       " 619: 4,\n",
       " 618: 1,\n",
       " 617: 1,\n",
       " 614: 22,\n",
       " 616: 2,\n",
       " 620: 31,\n",
       " 621: 10,\n",
       " 622: 6,\n",
       " 623: 22,\n",
       " 625: 1,\n",
       " 624: 7,\n",
       " 626: 4,\n",
       " 627: 9,\n",
       " 631: 21,\n",
       " 632: 3,\n",
       " 628: 1,\n",
       " 634: 7,\n",
       " 633: 3,\n",
       " 630: 2,\n",
       " 629: 1,\n",
       " 635: 23,\n",
       " 639: 26,\n",
       " 637: 61,\n",
       " 638: 47,\n",
       " 640: 3,\n",
       " 636: 2,\n",
       " 642: 7,\n",
       " 641: 12,\n",
       " 643: 7,\n",
       " 648: 3,\n",
       " 651: 37,\n",
       " 646: 6,\n",
       " 654: 6,\n",
       " 652: 1,\n",
       " 650: 3,\n",
       " 653: 2,\n",
       " 644: 8,\n",
       " 656: 12,\n",
       " 655: 7,\n",
       " 649: 6,\n",
       " 647: 7,\n",
       " 657: 4,\n",
       " 645: 2,\n",
       " 658: 29,\n",
       " 659: 5,\n",
       " 660: 14,\n",
       " 661: 17,\n",
       " 662: 5,\n",
       " 664: 11,\n",
       " 665: 1,\n",
       " 663: 1,\n",
       " 667: 4,\n",
       " 668: 2,\n",
       " 666: 14,\n",
       " 669: 6,\n",
       " 673: 38,\n",
       " 675: 17,\n",
       " 670: 35,\n",
       " 672: 14,\n",
       " 671: 4,\n",
       " 674: 16,\n",
       " 677: 70,\n",
       " 676: 17,\n",
       " 678: 4,\n",
       " 683: 1,\n",
       " 685: 2,\n",
       " 682: 1,\n",
       " 686: 44,\n",
       " 679: 2,\n",
       " 681: 1,\n",
       " 684: 3,\n",
       " 680: 43,\n",
       " 692: 14,\n",
       " 690: 4,\n",
       " 688: 9,\n",
       " 689: 23,\n",
       " 687: 19,\n",
       " 691: 6,\n",
       " 693: 8,\n",
       " 694: 7,\n",
       " 695: 1,\n",
       " 696: 34,\n",
       " 697: 23,\n",
       " 699: 49,\n",
       " 698: 3,\n",
       " 700: 9,\n",
       " 706: 18,\n",
       " 705: 9,\n",
       " 703: 23,\n",
       " 704: 3,\n",
       " 702: 3,\n",
       " 701: 30,\n",
       " 707: 17,\n",
       " 708: 3,\n",
       " 709: 12,\n",
       " 713: 20,\n",
       " 714: 18,\n",
       " 716: 43,\n",
       " 710: 2,\n",
       " 715: 29,\n",
       " 712: 12,\n",
       " 711: 8,\n",
       " 719: 7,\n",
       " 720: 21,\n",
       " 718: 6,\n",
       " 717: 9,\n",
       " 722: 31,\n",
       " 723: 17,\n",
       " 721: 3,\n",
       " 727: 16,\n",
       " 725: 2,\n",
       " 726: 5,\n",
       " 724: 5,\n",
       " 728: 23,\n",
       " 734: 36,\n",
       " 735: 2,\n",
       " 731: 14,\n",
       " 736: 47,\n",
       " 732: 4,\n",
       " 730: 6,\n",
       " 733: 4,\n",
       " 729: 20,\n",
       " 737: 4,\n",
       " 738: 32,\n",
       " 739: 9,\n",
       " 743: 6,\n",
       " 746: 20,\n",
       " 748: 13,\n",
       " 742: 2,\n",
       " 744: 3,\n",
       " 749: 8,\n",
       " 747: 2,\n",
       " 745: 4,\n",
       " 740: 17,\n",
       " 741: 1,\n",
       " 752: 6,\n",
       " 751: 1,\n",
       " 750: 4,\n",
       " 755: 8,\n",
       " 753: 2,\n",
       " 756: 10,\n",
       " 754: 7,\n",
       " 757: 2,\n",
       " 758: 26,\n",
       " 761: 4,\n",
       " 760: 5,\n",
       " 762: 2,\n",
       " 759: 12,\n",
       " 766: 3,\n",
       " 763: 3,\n",
       " 764: 11,\n",
       " 767: 3,\n",
       " 765: 7,\n",
       " 768: 6,\n",
       " 771: 12,\n",
       " 769: 39,\n",
       " 772: 28,\n",
       " 770: 6,\n",
       " 773: 20,\n",
       " 774: 14,\n",
       " 776: 2,\n",
       " 775: 3,\n",
       " 777: 3,\n",
       " 778: 11,\n",
       " 779: 21,\n",
       " 780: 5,\n",
       " 781: 21,\n",
       " 783: 5,\n",
       " 782: 28,\n",
       " 784: 27,\n",
       " 785: 11,\n",
       " 787: 2,\n",
       " 786: 13,\n",
       " 793: 5,\n",
       " 794: 42,\n",
       " 790: 6,\n",
       " 791: 5,\n",
       " 788: 3,\n",
       " 792: 2,\n",
       " 789: 11,\n",
       " 795: 2,\n",
       " 796: 4,\n",
       " 797: 33,\n",
       " 798: 4,\n",
       " 799: 5,\n",
       " 806: 33,\n",
       " 802: 6,\n",
       " 803: 5,\n",
       " 801: 15,\n",
       " 805: 11,\n",
       " 807: 13,\n",
       " 804: 6,\n",
       " 800: 6,\n",
       " 810: 19,\n",
       " 809: 12,\n",
       " 808: 1,\n",
       " 816: 10,\n",
       " 815: 2,\n",
       " 814: 2,\n",
       " 811: 12,\n",
       " 813: 10,\n",
       " 812: 2,\n",
       " 818: 13,\n",
       " 817: 11,\n",
       " 821: 7,\n",
       " 820: 8,\n",
       " 819: 12,\n",
       " 823: 4,\n",
       " 822: 6,\n",
       " 824: 8,\n",
       " 825: 19,\n",
       " 830: 5,\n",
       " 832: 33,\n",
       " 826: 4,\n",
       " 828: 1,\n",
       " 835: 21,\n",
       " 834: 10,\n",
       " 833: 2,\n",
       " 829: 29,\n",
       " 827: 28,\n",
       " 836: 1,\n",
       " 831: 32,\n",
       " 840: 26,\n",
       " 837: 12,\n",
       " 838: 6,\n",
       " 839: 5,\n",
       " 841: 2,\n",
       " 842: 13,\n",
       " 843: 6,\n",
       " 845: 1,\n",
       " 844: 1,\n",
       " 846: 13,\n",
       " 849: 1,\n",
       " 847: 9,\n",
       " 848: 6,\n",
       " 853: 7,\n",
       " 850: 5,\n",
       " 851: 2,\n",
       " 854: 8,\n",
       " 852: 1,\n",
       " 858: 7,\n",
       " 856: 27,\n",
       " 855: 18,\n",
       " 857: 1,\n",
       " 861: 18,\n",
       " 859: 2,\n",
       " 860: 3,\n",
       " 862: 2,\n",
       " 864: 74,\n",
       " 863: 4,\n",
       " 867: 1,\n",
       " 870: 4,\n",
       " 865: 3,\n",
       " 866: 25,\n",
       " 871: 8,\n",
       " 868: 8,\n",
       " 869: 4,\n",
       " 872: 1,\n",
       " 873: 17,\n",
       " 875: 1,\n",
       " 876: 1,\n",
       " 879: 5,\n",
       " 874: 4,\n",
       " 878: 5,\n",
       " 877: 12,\n",
       " 880: 1,\n",
       " 882: 7,\n",
       " 884: 5,\n",
       " 883: 2,\n",
       " 881: 1,\n",
       " 885: 2,\n",
       " 887: 9,\n",
       " 888: 7,\n",
       " 886: 16,\n",
       " 889: 6,\n",
       " 890: 22,\n",
       " 894: 4,\n",
       " 891: 6,\n",
       " 893: 10,\n",
       " 892: 1,\n",
       " 896: 6,\n",
       " 897: 19,\n",
       " 895: 6,\n",
       " 898: 22,\n",
       " 899: 9,\n",
       " 900: 29,\n",
       " 901: 3,\n",
       " 915: 6,\n",
       " 905: 2,\n",
       " 911: 4,\n",
       " 904: 4,\n",
       " 909: 3,\n",
       " 913: 4,\n",
       " 902: 1,\n",
       " 914: 2,\n",
       " 907: 15,\n",
       " 910: 1,\n",
       " 912: 17,\n",
       " 903: 8,\n",
       " 908: 33,\n",
       " 906: 8,\n",
       " 916: 4,\n",
       " 918: 3,\n",
       " 917: 13,\n",
       " 919: 23,\n",
       " 921: 7,\n",
       " 920: 10,\n",
       " 923: 5,\n",
       " 922: 7,\n",
       " 928: 2,\n",
       " 925: 6,\n",
       " 924: 1,\n",
       " 929: 1,\n",
       " 931: 1,\n",
       " 927: 15,\n",
       " 926: 1,\n",
       " 930: 5,\n",
       " 938: 14,\n",
       " 937: 1,\n",
       " 935: 3,\n",
       " 936: 5,\n",
       " 934: 31,\n",
       " 932: 5,\n",
       " 933: 3,\n",
       " 939: 14,\n",
       " 944: 8,\n",
       " 940: 11,\n",
       " 942: 3,\n",
       " 941: 2,\n",
       " 943: 6,\n",
       " 945: 3,\n",
       " 948: 10,\n",
       " 949: 2,\n",
       " 946: 3,\n",
       " 947: 12,\n",
       " 950: 4,\n",
       " 951: 3,\n",
       " 952: 6,\n",
       " 953: 3,\n",
       " 954: 2,\n",
       " 957: 26,\n",
       " 956: 9,\n",
       " 955: 14,\n",
       " 958: 3,\n",
       " 960: 2,\n",
       " 959: 2,\n",
       " 964: 5,\n",
       " 961: 2,\n",
       " 962: 2,\n",
       " 963: 4,\n",
       " 965: 3,\n",
       " 967: 1,\n",
       " 966: 6,\n",
       " 968: 2,\n",
       " 973: 39,\n",
       " 970: 14,\n",
       " 972: 8,\n",
       " 969: 6,\n",
       " 971: 7,\n",
       " 975: 3,\n",
       " 974: 3,\n",
       " 980: 6,\n",
       " 978: 1,\n",
       " 979: 1,\n",
       " 981: 1,\n",
       " 977: 12,\n",
       " 976: 3,\n",
       " 982: 3,\n",
       " 991: 3,\n",
       " 988: 13,\n",
       " 987: 12,\n",
       " 989: 2,\n",
       " 990: 27,\n",
       " 984: 2,\n",
       " 986: 2,\n",
       " 983: 2,\n",
       " 985: 2,\n",
       " 992: 1,\n",
       " 995: 6,\n",
       " 994: 12,\n",
       " 993: 21,\n",
       " 996: 20,\n",
       " 997: 1,\n",
       " 999: 2,\n",
       " 1001: 6,\n",
       " ...}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.cfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8f_mMaGqPIo6",
    "outputId": "d75e61a5-94ff-49e0-c320-b5dbe0acefc5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{18: 181,\n",
       " 6: 373,\n",
       " 5: 294,\n",
       " 3: 16,\n",
       " 15: 9,\n",
       " 7: 387,\n",
       " 11: 6,\n",
       " 1: 1290,\n",
       " 14: 5,\n",
       " 8: 25,\n",
       " 16: 4,\n",
       " 19: 316,\n",
       " 9: 202,\n",
       " 10: 2,\n",
       " 13: 14,\n",
       " 2: 112,\n",
       " 17: 1049,\n",
       " 4: 3,\n",
       " 0: 12,\n",
       " 12: 3,\n",
       " 21: 36,\n",
       " 20: 307,\n",
       " 36: 487,\n",
       " 43: 525,\n",
       " 37: 347,\n",
       " 31: 73,\n",
       " 39: 16,\n",
       " 41: 498,\n",
       " 24: 348,\n",
       " 42: 1882,\n",
       " 30: 37,\n",
       " 35: 64,\n",
       " 25: 3,\n",
       " 27: 95,\n",
       " 38: 18,\n",
       " 26: 96,\n",
       " 22: 6,\n",
       " 32: 114,\n",
       " 33: 5,\n",
       " 34: 174,\n",
       " 29: 116,\n",
       " 28: 1,\n",
       " 40: 2,\n",
       " 23: 54,\n",
       " 47: 23,\n",
       " 57: 90,\n",
       " 53: 194,\n",
       " 45: 83,\n",
       " 44: 8,\n",
       " 56: 183,\n",
       " 55: 70,\n",
       " 46: 367,\n",
       " 54: 93,\n",
       " 51: 35,\n",
       " 48: 286,\n",
       " 50: 110,\n",
       " 52: 16,\n",
       " 49: 78,\n",
       " 62: 142,\n",
       " 58: 92,\n",
       " 61: 7,\n",
       " 59: 26,\n",
       " 60: 16,\n",
       " 64: 147,\n",
       " 63: 111,\n",
       " 72: 77,\n",
       " 67: 3,\n",
       " 69: 799,\n",
       " 71: 2,\n",
       " 65: 12,\n",
       " 68: 10,\n",
       " 66: 11,\n",
       " 70: 101,\n",
       " 75: 43,\n",
       " 76: 141,\n",
       " 82: 135,\n",
       " 73: 32,\n",
       " 78: 158,\n",
       " 74: 50,\n",
       " 80: 194,\n",
       " 84: 74,\n",
       " 83: 21,\n",
       " 81: 183,\n",
       " 77: 113,\n",
       " 79: 273,\n",
       " 86: 27,\n",
       " 85: 29,\n",
       " 89: 145,\n",
       " 91: 566,\n",
       " 90: 25,\n",
       " 88: 10,\n",
       " 87: 11,\n",
       " 97: 46,\n",
       " 96: 68,\n",
       " 95: 9,\n",
       " 100: 40,\n",
       " 103: 9,\n",
       " 92: 5,\n",
       " 104: 11,\n",
       " 99: 63,\n",
       " 93: 21,\n",
       " 94: 13,\n",
       " 98: 68,\n",
       " 102: 173,\n",
       " 101: 46,\n",
       " 106: 3,\n",
       " 109: 15,\n",
       " 111: 200,\n",
       " 107: 1,\n",
       " 110: 2,\n",
       " 108: 30,\n",
       " 105: 2,\n",
       " 113: 9,\n",
       " 112: 44,\n",
       " 115: 245,\n",
       " 116: 10,\n",
       " 117: 27,\n",
       " 114: 129,\n",
       " 121: 107,\n",
       " 127: 33,\n",
       " 119: 303,\n",
       " 125: 7,\n",
       " 133: 637,\n",
       " 130: 43,\n",
       " 139: 45,\n",
       " 135: 13,\n",
       " 124: 1,\n",
       " 141: 71,\n",
       " 140: 47,\n",
       " 123: 11,\n",
       " 142: 140,\n",
       " 122: 69,\n",
       " 126: 4,\n",
       " 118: 110,\n",
       " 129: 14,\n",
       " 136: 36,\n",
       " 137: 214,\n",
       " 138: 83,\n",
       " 134: 36,\n",
       " 132: 10,\n",
       " 131: 6,\n",
       " 128: 77,\n",
       " 120: 324,\n",
       " 144: 39,\n",
       " 145: 3,\n",
       " 143: 5,\n",
       " 146: 6,\n",
       " 152: 105,\n",
       " 151: 4,\n",
       " 148: 15,\n",
       " 147: 587,\n",
       " 149: 26,\n",
       " 150: 6,\n",
       " 153: 7,\n",
       " 159: 150,\n",
       " 158: 67,\n",
       " 163: 57,\n",
       " 154: 68,\n",
       " 160: 69,\n",
       " 164: 7,\n",
       " 156: 62,\n",
       " 162: 9,\n",
       " 165: 68,\n",
       " 155: 31,\n",
       " 161: 131,\n",
       " 157: 3,\n",
       " 168: 68,\n",
       " 170: 26,\n",
       " 167: 77,\n",
       " 166: 11,\n",
       " 169: 45,\n",
       " 181: 342,\n",
       " 172: 65,\n",
       " 173: 74,\n",
       " 177: 78,\n",
       " 178: 29,\n",
       " 184: 48,\n",
       " 175: 10,\n",
       " 183: 70,\n",
       " 182: 3,\n",
       " 176: 2,\n",
       " 180: 64,\n",
       " 179: 11,\n",
       " 171: 99,\n",
       " 174: 4,\n",
       " 187: 79,\n",
       " 190: 63,\n",
       " 189: 105,\n",
       " 191: 145,\n",
       " 188: 6,\n",
       " 185: 15,\n",
       " 186: 12,\n",
       " 193: 17,\n",
       " 195: 50,\n",
       " 196: 18,\n",
       " 194: 6,\n",
       " 192: 23,\n",
       " 197: 104,\n",
       " 199: 6,\n",
       " 198: 6,\n",
       " 200: 4,\n",
       " 207: 158,\n",
       " 205: 136,\n",
       " 202: 47,\n",
       " 209: 153,\n",
       " 203: 9,\n",
       " 206: 55,\n",
       " 204: 115,\n",
       " 208: 50,\n",
       " 201: 336,\n",
       " 211: 15,\n",
       " 213: 4,\n",
       " 216: 599,\n",
       " 221: 19,\n",
       " 222: 4,\n",
       " 220: 191,\n",
       " 212: 9,\n",
       " 217: 62,\n",
       " 210: 34,\n",
       " 219: 20,\n",
       " 214: 6,\n",
       " 218: 79,\n",
       " 223: 1,\n",
       " 215: 6,\n",
       " 224: 100,\n",
       " 236: 29,\n",
       " 233: 12,\n",
       " 232: 50,\n",
       " 226: 73,\n",
       " 231: 5,\n",
       " 227: 8,\n",
       " 237: 16,\n",
       " 234: 279,\n",
       " 225: 5,\n",
       " 239: 3,\n",
       " 235: 81,\n",
       " 240: 179,\n",
       " 228: 15,\n",
       " 229: 10,\n",
       " 230: 20,\n",
       " 238: 3,\n",
       " 242: 23,\n",
       " 244: 252,\n",
       " 245: 167,\n",
       " 246: 37,\n",
       " 248: 32,\n",
       " 243: 4,\n",
       " 241: 6,\n",
       " 247: 12,\n",
       " 255: 60,\n",
       " 253: 40,\n",
       " 251: 3,\n",
       " 250: 7,\n",
       " 254: 167,\n",
       " 252: 8,\n",
       " 249: 13,\n",
       " 269: 55,\n",
       " 256: 29,\n",
       " 267: 10,\n",
       " 265: 107,\n",
       " 272: 88,\n",
       " 262: 29,\n",
       " 268: 42,\n",
       " 259: 7,\n",
       " 258: 144,\n",
       " 264: 39,\n",
       " 261: 20,\n",
       " 266: 11,\n",
       " 271: 16,\n",
       " 257: 43,\n",
       " 263: 50,\n",
       " 260: 18,\n",
       " 270: 9,\n",
       " 279: 99,\n",
       " 275: 35,\n",
       " 278: 4,\n",
       " 273: 104,\n",
       " 276: 34,\n",
       " 277: 23,\n",
       " 274: 8,\n",
       " 280: 29,\n",
       " 285: 13,\n",
       " 282: 5,\n",
       " 286: 32,\n",
       " 281: 67,\n",
       " 284: 8,\n",
       " 283: 17,\n",
       " 291: 54,\n",
       " 294: 24,\n",
       " 298: 2,\n",
       " 297: 50,\n",
       " 287: 38,\n",
       " 295: 7,\n",
       " 288: 2,\n",
       " 293: 59,\n",
       " 292: 1,\n",
       " 296: 21,\n",
       " 289: 41,\n",
       " 290: 51,\n",
       " 299: 149,\n",
       " 305: 85,\n",
       " 300: 14,\n",
       " 307: 8,\n",
       " 301: 3,\n",
       " 309: 22,\n",
       " 308: 33,\n",
       " 306: 29,\n",
       " 302: 5,\n",
       " 310: 1,\n",
       " 304: 124,\n",
       " 303: 15,\n",
       " 317: 22,\n",
       " 314: 17,\n",
       " 318: 11,\n",
       " 313: 1,\n",
       " 315: 4,\n",
       " 316: 35,\n",
       " 311: 92,\n",
       " 319: 6,\n",
       " 312: 2,\n",
       " 333: 96,\n",
       " 324: 7,\n",
       " 329: 2,\n",
       " 327: 15,\n",
       " 330: 4,\n",
       " 332: 43,\n",
       " 331: 2,\n",
       " 321: 7,\n",
       " 323: 1,\n",
       " 328: 3,\n",
       " 326: 5,\n",
       " 325: 6,\n",
       " 320: 5,\n",
       " 322: 2,\n",
       " 334: 13,\n",
       " 335: 7,\n",
       " 336: 33,\n",
       " 339: 19,\n",
       " 338: 6,\n",
       " 337: 26,\n",
       " 342: 5,\n",
       " 347: 6,\n",
       " 345: 78,\n",
       " 348: 76,\n",
       " 344: 25,\n",
       " 340: 20,\n",
       " 346: 6,\n",
       " 343: 22,\n",
       " 341: 18,\n",
       " 349: 1,\n",
       " 350: 166,\n",
       " 351: 18,\n",
       " 352: 16,\n",
       " 356: 23,\n",
       " 355: 5,\n",
       " 357: 32,\n",
       " 361: 149,\n",
       " 359: 1,\n",
       " 360: 89,\n",
       " 354: 11,\n",
       " 353: 1,\n",
       " 358: 16,\n",
       " 370: 38,\n",
       " 362: 25,\n",
       " 363: 13,\n",
       " 368: 5,\n",
       " 369: 224,\n",
       " 364: 1,\n",
       " 367: 4,\n",
       " 366: 13,\n",
       " 365: 20,\n",
       " 373: 34,\n",
       " 375: 42,\n",
       " 371: 11,\n",
       " 376: 16,\n",
       " 372: 26,\n",
       " 374: 84,\n",
       " 377: 176,\n",
       " 378: 28,\n",
       " 380: 15,\n",
       " 379: 22,\n",
       " 381: 25,\n",
       " 383: 6,\n",
       " 384: 14,\n",
       " 382: 6,\n",
       " 386: 14,\n",
       " 385: 5,\n",
       " 389: 2,\n",
       " 388: 3,\n",
       " 387: 3,\n",
       " 390: 67,\n",
       " 396: 4,\n",
       " 400: 53,\n",
       " 397: 19,\n",
       " 399: 11,\n",
       " 398: 4,\n",
       " 394: 16,\n",
       " 395: 56,\n",
       " 392: 11,\n",
       " 391: 11,\n",
       " 393: 16,\n",
       " 402: 4,\n",
       " 404: 18,\n",
       " 405: 5,\n",
       " 403: 33,\n",
       " 401: 20,\n",
       " 406: 77,\n",
       " 407: 32,\n",
       " 408: 7,\n",
       " 409: 3,\n",
       " 416: 16,\n",
       " 410: 5,\n",
       " 414: 1,\n",
       " 413: 3,\n",
       " 411: 19,\n",
       " 415: 11,\n",
       " 412: 3,\n",
       " 421: 23,\n",
       " 419: 7,\n",
       " 417: 17,\n",
       " 420: 18,\n",
       " 422: 6,\n",
       " 418: 31,\n",
       " 428: 18,\n",
       " 427: 1,\n",
       " 430: 26,\n",
       " 425: 16,\n",
       " 424: 33,\n",
       " 423: 24,\n",
       " 426: 30,\n",
       " 429: 23,\n",
       " 432: 10,\n",
       " 431: 2,\n",
       " 433: 5,\n",
       " 441: 17,\n",
       " 435: 3,\n",
       " 440: 2,\n",
       " 434: 81,\n",
       " 438: 2,\n",
       " 442: 14,\n",
       " 439: 12,\n",
       " 437: 8,\n",
       " 436: 22,\n",
       " 445: 2,\n",
       " 443: 9,\n",
       " 444: 24,\n",
       " 447: 56,\n",
       " 446: 3,\n",
       " 448: 13,\n",
       " 450: 5,\n",
       " 451: 12,\n",
       " 449: 7,\n",
       " 454: 8,\n",
       " 455: 9,\n",
       " 452: 18,\n",
       " 456: 41,\n",
       " 453: 2,\n",
       " 461: 108,\n",
       " 458: 24,\n",
       " 460: 14,\n",
       " 459: 9,\n",
       " 457: 18,\n",
       " 462: 15,\n",
       " 464: 22,\n",
       " 467: 13,\n",
       " 465: 16,\n",
       " 469: 1,\n",
       " 463: 38,\n",
       " 468: 79,\n",
       " 470: 3,\n",
       " 471: 25,\n",
       " 466: 2,\n",
       " 472: 12,\n",
       " 473: 1,\n",
       " 480: 28,\n",
       " 475: 15,\n",
       " 478: 11,\n",
       " 479: 3,\n",
       " 474: 3,\n",
       " 477: 2,\n",
       " 476: 1,\n",
       " 482: 22,\n",
       " 483: 12,\n",
       " 481: 21,\n",
       " 485: 9,\n",
       " 484: 6,\n",
       " 488: 33,\n",
       " 486: 30,\n",
       " 487: 1,\n",
       " 489: 1,\n",
       " 491: 45,\n",
       " 494: 4,\n",
       " 493: 8,\n",
       " 490: 14,\n",
       " 495: 6,\n",
       " 492: 6,\n",
       " 500: 24,\n",
       " 499: 24,\n",
       " 498: 17,\n",
       " 497: 11,\n",
       " 496: 16,\n",
       " 501: 34,\n",
       " 502: 14,\n",
       " 504: 7,\n",
       " 503: 11,\n",
       " 505: 14,\n",
       " 508: 11,\n",
       " 506: 3,\n",
       " 507: 3,\n",
       " 510: 11,\n",
       " 509: 4,\n",
       " 511: 33,\n",
       " 512: 11,\n",
       " 514: 6,\n",
       " 513: 4,\n",
       " 515: 104,\n",
       " 516: 76,\n",
       " 517: 9,\n",
       " 521: 4,\n",
       " 522: 19,\n",
       " 518: 110,\n",
       " 520: 4,\n",
       " 523: 5,\n",
       " 519: 5,\n",
       " 524: 22,\n",
       " 525: 17,\n",
       " 526: 73,\n",
       " 527: 6,\n",
       " 531: 4,\n",
       " 532: 12,\n",
       " 529: 12,\n",
       " 528: 2,\n",
       " 530: 8,\n",
       " 533: 3,\n",
       " 535: 7,\n",
       " 536: 16,\n",
       " 534: 7,\n",
       " 549: 24,\n",
       " 539: 2,\n",
       " 543: 3,\n",
       " 540: 20,\n",
       " 544: 5,\n",
       " 537: 2,\n",
       " 546: 42,\n",
       " 545: 52,\n",
       " 541: 6,\n",
       " 538: 1,\n",
       " 547: 7,\n",
       " 548: 6,\n",
       " 542: 4,\n",
       " 550: 17,\n",
       " 551: 2,\n",
       " 552: 13,\n",
       " 557: 56,\n",
       " 554: 1,\n",
       " 553: 1,\n",
       " 560: 4,\n",
       " 558: 14,\n",
       " 555: 5,\n",
       " 559: 2,\n",
       " 556: 7,\n",
       " 561: 11,\n",
       " 562: 26,\n",
       " 564: 9,\n",
       " 563: 10,\n",
       " 566: 52,\n",
       " 565: 13,\n",
       " 567: 22,\n",
       " 568: 1,\n",
       " 569: 12,\n",
       " 571: 22,\n",
       " 573: 1,\n",
       " 575: 14,\n",
       " 574: 1,\n",
       " 572: 37,\n",
       " 576: 24,\n",
       " 570: 1,\n",
       " 580: 29,\n",
       " 578: 7,\n",
       " 579: 2,\n",
       " 577: 1,\n",
       " 581: 1,\n",
       " 587: 1,\n",
       " 590: 4,\n",
       " 585: 3,\n",
       " 589: 2,\n",
       " 588: 1,\n",
       " 583: 21,\n",
       " 586: 1,\n",
       " 582: 12,\n",
       " 591: 23,\n",
       " 584: 5,\n",
       " 592: 3,\n",
       " 594: 3,\n",
       " 600: 1,\n",
       " 601: 1,\n",
       " 593: 4,\n",
       " 599: 1,\n",
       " 602: 1,\n",
       " 597: 13,\n",
       " 595: 2,\n",
       " 603: 1,\n",
       " 598: 3,\n",
       " 596: 3,\n",
       " 604: 19,\n",
       " 606: 55,\n",
       " 605: 11,\n",
       " 607: 4,\n",
       " 612: 6,\n",
       " 609: 24,\n",
       " 608: 3,\n",
       " 611: 6,\n",
       " 610: 3,\n",
       " 613: 1,\n",
       " 615: 25,\n",
       " 619: 4,\n",
       " 618: 1,\n",
       " 617: 1,\n",
       " 614: 20,\n",
       " 616: 2,\n",
       " 620: 29,\n",
       " 621: 10,\n",
       " 622: 5,\n",
       " 623: 22,\n",
       " 625: 1,\n",
       " 624: 6,\n",
       " 626: 4,\n",
       " 627: 9,\n",
       " 631: 21,\n",
       " 632: 3,\n",
       " 628: 1,\n",
       " 634: 7,\n",
       " 633: 3,\n",
       " 630: 2,\n",
       " 629: 1,\n",
       " 635: 23,\n",
       " 639: 26,\n",
       " 637: 56,\n",
       " 638: 45,\n",
       " 640: 3,\n",
       " 636: 2,\n",
       " 642: 7,\n",
       " 641: 12,\n",
       " 643: 7,\n",
       " 648: 3,\n",
       " 651: 37,\n",
       " 646: 5,\n",
       " 654: 6,\n",
       " 652: 1,\n",
       " 650: 3,\n",
       " 653: 2,\n",
       " 644: 8,\n",
       " 656: 12,\n",
       " 655: 7,\n",
       " 649: 6,\n",
       " 647: 7,\n",
       " 657: 4,\n",
       " 645: 2,\n",
       " 658: 29,\n",
       " 659: 5,\n",
       " 660: 14,\n",
       " 661: 17,\n",
       " 662: 5,\n",
       " 664: 11,\n",
       " 665: 1,\n",
       " 663: 1,\n",
       " 667: 4,\n",
       " 668: 2,\n",
       " 666: 13,\n",
       " 669: 6,\n",
       " 673: 37,\n",
       " 675: 17,\n",
       " 670: 35,\n",
       " 672: 11,\n",
       " 671: 4,\n",
       " 674: 16,\n",
       " 677: 69,\n",
       " 676: 17,\n",
       " 678: 4,\n",
       " 683: 1,\n",
       " 685: 2,\n",
       " 682: 1,\n",
       " 686: 40,\n",
       " 679: 2,\n",
       " 681: 1,\n",
       " 684: 3,\n",
       " 680: 43,\n",
       " 692: 13,\n",
       " 690: 4,\n",
       " 688: 9,\n",
       " 689: 23,\n",
       " 687: 19,\n",
       " 691: 6,\n",
       " 693: 8,\n",
       " 694: 7,\n",
       " 695: 1,\n",
       " 696: 34,\n",
       " 697: 23,\n",
       " 699: 49,\n",
       " 698: 3,\n",
       " 700: 9,\n",
       " 706: 18,\n",
       " 705: 9,\n",
       " 703: 22,\n",
       " 704: 3,\n",
       " 702: 3,\n",
       " 701: 30,\n",
       " 707: 17,\n",
       " 708: 3,\n",
       " 709: 11,\n",
       " 713: 17,\n",
       " 714: 18,\n",
       " 716: 43,\n",
       " 710: 2,\n",
       " 715: 28,\n",
       " 712: 10,\n",
       " 711: 8,\n",
       " 719: 7,\n",
       " 720: 21,\n",
       " 718: 6,\n",
       " 717: 9,\n",
       " 722: 30,\n",
       " 723: 17,\n",
       " 721: 3,\n",
       " 727: 16,\n",
       " 725: 2,\n",
       " 726: 5,\n",
       " 724: 5,\n",
       " 728: 23,\n",
       " 734: 36,\n",
       " 735: 2,\n",
       " 731: 14,\n",
       " 736: 44,\n",
       " 732: 4,\n",
       " 730: 6,\n",
       " 733: 4,\n",
       " 729: 20,\n",
       " 737: 4,\n",
       " 738: 32,\n",
       " 739: 9,\n",
       " 743: 5,\n",
       " 746: 20,\n",
       " 748: 13,\n",
       " 742: 2,\n",
       " 744: 3,\n",
       " 749: 8,\n",
       " 747: 2,\n",
       " 745: 4,\n",
       " 740: 17,\n",
       " 741: 1,\n",
       " 752: 6,\n",
       " 751: 1,\n",
       " 750: 4,\n",
       " 755: 8,\n",
       " 753: 2,\n",
       " 756: 10,\n",
       " 754: 7,\n",
       " 757: 2,\n",
       " 758: 26,\n",
       " 761: 4,\n",
       " 760: 5,\n",
       " 762: 2,\n",
       " 759: 12,\n",
       " 766: 3,\n",
       " 763: 3,\n",
       " 764: 10,\n",
       " 767: 3,\n",
       " 765: 7,\n",
       " 768: 6,\n",
       " 771: 12,\n",
       " 769: 37,\n",
       " 772: 28,\n",
       " 770: 6,\n",
       " 773: 20,\n",
       " 774: 14,\n",
       " 776: 2,\n",
       " 775: 3,\n",
       " 777: 3,\n",
       " 778: 10,\n",
       " 779: 21,\n",
       " 780: 5,\n",
       " 781: 19,\n",
       " 783: 5,\n",
       " 782: 28,\n",
       " 784: 26,\n",
       " 785: 11,\n",
       " 787: 2,\n",
       " 786: 13,\n",
       " 793: 5,\n",
       " 794: 41,\n",
       " 790: 5,\n",
       " 791: 5,\n",
       " 788: 3,\n",
       " 792: 2,\n",
       " 789: 11,\n",
       " 795: 2,\n",
       " 796: 4,\n",
       " 797: 33,\n",
       " 798: 4,\n",
       " 799: 5,\n",
       " 806: 33,\n",
       " 802: 6,\n",
       " 803: 5,\n",
       " 801: 15,\n",
       " 805: 11,\n",
       " 807: 13,\n",
       " 804: 6,\n",
       " 800: 5,\n",
       " 810: 19,\n",
       " 809: 9,\n",
       " 808: 1,\n",
       " 816: 9,\n",
       " 815: 2,\n",
       " 814: 2,\n",
       " 811: 11,\n",
       " 813: 10,\n",
       " 812: 2,\n",
       " 818: 13,\n",
       " 817: 11,\n",
       " 821: 7,\n",
       " 820: 7,\n",
       " 819: 12,\n",
       " 823: 4,\n",
       " 822: 6,\n",
       " 824: 8,\n",
       " 825: 19,\n",
       " 830: 5,\n",
       " 832: 33,\n",
       " 826: 4,\n",
       " 828: 1,\n",
       " 835: 19,\n",
       " 834: 10,\n",
       " 833: 2,\n",
       " 829: 28,\n",
       " 827: 27,\n",
       " 836: 1,\n",
       " 831: 32,\n",
       " 840: 26,\n",
       " 837: 12,\n",
       " 838: 6,\n",
       " 839: 5,\n",
       " 841: 2,\n",
       " 842: 12,\n",
       " 843: 6,\n",
       " 845: 1,\n",
       " 844: 1,\n",
       " 846: 11,\n",
       " 849: 1,\n",
       " 847: 9,\n",
       " 848: 6,\n",
       " 853: 7,\n",
       " 850: 5,\n",
       " 851: 2,\n",
       " 854: 7,\n",
       " 852: 1,\n",
       " 858: 7,\n",
       " 856: 27,\n",
       " 855: 18,\n",
       " 857: 1,\n",
       " 861: 18,\n",
       " 859: 2,\n",
       " 860: 3,\n",
       " 862: 2,\n",
       " 864: 74,\n",
       " 863: 4,\n",
       " 867: 1,\n",
       " 870: 3,\n",
       " 865: 3,\n",
       " 866: 24,\n",
       " 871: 8,\n",
       " 868: 8,\n",
       " 869: 4,\n",
       " 872: 1,\n",
       " 873: 16,\n",
       " 875: 1,\n",
       " 876: 1,\n",
       " 879: 5,\n",
       " 874: 4,\n",
       " 878: 5,\n",
       " 877: 12,\n",
       " 880: 1,\n",
       " 882: 7,\n",
       " 884: 4,\n",
       " 883: 2,\n",
       " 881: 1,\n",
       " 885: 1,\n",
       " 887: 9,\n",
       " 888: 7,\n",
       " 886: 16,\n",
       " 889: 6,\n",
       " 890: 21,\n",
       " 894: 4,\n",
       " 891: 6,\n",
       " 893: 10,\n",
       " 892: 1,\n",
       " 896: 6,\n",
       " 897: 19,\n",
       " 895: 6,\n",
       " 898: 21,\n",
       " 899: 8,\n",
       " 900: 26,\n",
       " 901: 3,\n",
       " 915: 6,\n",
       " 905: 2,\n",
       " 911: 4,\n",
       " 904: 4,\n",
       " 909: 3,\n",
       " 913: 4,\n",
       " 902: 1,\n",
       " 914: 2,\n",
       " 907: 15,\n",
       " 910: 1,\n",
       " 912: 17,\n",
       " 903: 8,\n",
       " 908: 33,\n",
       " 906: 8,\n",
       " 916: 4,\n",
       " 918: 3,\n",
       " 917: 13,\n",
       " 919: 23,\n",
       " 921: 7,\n",
       " 920: 10,\n",
       " 923: 4,\n",
       " 922: 7,\n",
       " 928: 2,\n",
       " 925: 6,\n",
       " 924: 1,\n",
       " 929: 1,\n",
       " 931: 1,\n",
       " 927: 15,\n",
       " 926: 1,\n",
       " 930: 5,\n",
       " 938: 13,\n",
       " 937: 1,\n",
       " 935: 3,\n",
       " 936: 5,\n",
       " 934: 31,\n",
       " 932: 5,\n",
       " 933: 2,\n",
       " 939: 13,\n",
       " 944: 2,\n",
       " 940: 9,\n",
       " 942: 3,\n",
       " 941: 2,\n",
       " 943: 6,\n",
       " 945: 3,\n",
       " 948: 10,\n",
       " 949: 2,\n",
       " 946: 3,\n",
       " 947: 12,\n",
       " 950: 4,\n",
       " 951: 3,\n",
       " 952: 6,\n",
       " 953: 3,\n",
       " 954: 2,\n",
       " 957: 26,\n",
       " 956: 9,\n",
       " 955: 14,\n",
       " 958: 3,\n",
       " 960: 2,\n",
       " 959: 2,\n",
       " 964: 5,\n",
       " 961: 2,\n",
       " 962: 2,\n",
       " 963: 4,\n",
       " 965: 3,\n",
       " 967: 1,\n",
       " 966: 6,\n",
       " 968: 2,\n",
       " 973: 37,\n",
       " 970: 13,\n",
       " 972: 8,\n",
       " 969: 6,\n",
       " 971: 7,\n",
       " 975: 3,\n",
       " 974: 3,\n",
       " 980: 6,\n",
       " 978: 1,\n",
       " 979: 1,\n",
       " 981: 1,\n",
       " 977: 12,\n",
       " 976: 3,\n",
       " 982: 3,\n",
       " 991: 3,\n",
       " 988: 13,\n",
       " 987: 12,\n",
       " 989: 2,\n",
       " 990: 25,\n",
       " 984: 2,\n",
       " 986: 2,\n",
       " 983: 2,\n",
       " 985: 2,\n",
       " 992: 1,\n",
       " 995: 6,\n",
       " 994: 12,\n",
       " 993: 17,\n",
       " 996: 20,\n",
       " 997: 1,\n",
       " 999: 2,\n",
       " 1001: 6,\n",
       " ...}"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dictionary.dfs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "D8trSPV_PIo6",
    "outputId": "ad62cd58-b52e-4b8d-b9a3-e56887039e9c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['follow', 'the', 'white', 'rabbit']\n",
      "[194, 42, 248, 246]\n"
     ]
    }
   ],
   "source": [
    "sentence = 'follow the white rabbit'\n",
    "new_tokens = simple_preprocess(sentence)\n",
    "ids = dictionary.doc2idx(new_tokens)\n",
    "print(new_tokens)\n",
    "print(ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "jTnw8JNXPIo6"
   },
   "outputs": [],
   "source": [
    "special_tokens = {'[PAD]': 0, '[UNK]': 1}\n",
    "dictionary.patch_with_special_tokens(special_tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "2y0NKpAuPIo7"
   },
   "outputs": [],
   "source": [
    "def get_rare_ids(dictionary, min_freq):\n",
    "    rare_ids = [t[0] for t in dictionary.cfs.items() if t[1] < min_freq]\n",
    "    return rare_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "rO9cRL8fPIo7"
   },
   "outputs": [],
   "source": [
    "def make_vocab(sentences, folder=None, special_tokens=None, vocab_size=None, min_freq=None):\n",
    "    if folder is not None:\n",
    "        if not os.path.exists(folder):\n",
    "            os.mkdir(folder)\n",
    "\n",
    "    # tokenizes the sentences and create a Dictionary\n",
    "    tokens = [simple_preprocess(sent) for sent in sentences]\n",
    "    dictionary = corpora.Dictionary(tokens)\n",
    "    # keeps only the most frequent words (vocab size)\n",
    "    if vocab_size is not None:\n",
    "        dictionary.filter_extremes(keep_n=vocab_size)\n",
    "    # removes rare words (in case the vocab size still\n",
    "    # includes words with low frequency)\n",
    "    if min_freq is not None:\n",
    "        rare_tokens = get_rare_ids(dictionary, min_freq)\n",
    "        dictionary.filter_tokens(bad_ids=rare_tokens)\n",
    "    # gets the whole list of tokens and frequencies\n",
    "    items = dictionary.cfs.items()\n",
    "    # sorts the tokens in descending order\n",
    "    words = [dictionary[t[0]] for t in sorted(dictionary.cfs.items(), key=lambda t: -t[1])]\n",
    "    # prepends special tokens, if any\n",
    "    if special_tokens is not None:\n",
    "        to_add = []\n",
    "        for special_token in special_tokens:\n",
    "            if special_token not in words:\n",
    "                to_add.append(special_token)\n",
    "        words = to_add + words\n",
    "\n",
    "    with open(os.path.join(folder, 'vocab.txt'), 'w') as f:\n",
    "        for word in words:\n",
    "            f.write(f'{word}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "id": "3x1exiX0PIo7"
   },
   "outputs": [],
   "source": [
    "make_vocab(train_dataset['sentence'], 'our_vocab/', special_tokens=['[PAD]', '[UNK]', '[SEP]', '[CLS]', '[MASK]'], min_freq=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7Cq5LmV6PIo7"
   },
   "source": [
    "## HugginFace's Tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "id": "AbW7Cd8jPIo8"
   },
   "outputs": [],
   "source": [
    "from transformers import BertTokenizer\n",
    "\n",
    "tokenizer = BertTokenizer('our_vocab/vocab.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nhxCrkJjPIo8",
    "outputId": "9eca4ff5-ac7a-407f-87f2-ae2f4fddd9a2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['follow', 'the', 'white', 'rabbit', '[UNK]']"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_sentence = 'follow the white rabbit neo'\n",
    "new_tokens = tokenizer.tokenize(new_sentence)\n",
    "new_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "V6DlCKGxPIo8",
    "outputId": "9594a418-42a3-4014-f4ff-0a9cd7a728d6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[864, 5, 224, 190, 1]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_ids = tokenizer.convert_tokens_to_ids(new_tokens)\n",
    "new_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "06gmmhDvPIo9",
    "outputId": "c56341b6-22a5-443d-f6bc-66df31fbfcb4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[3, 864, 5, 224, 190, 1, 2]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_ids = tokenizer.encode(new_sentence)\n",
    "new_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "98Ccyt_LPIo9",
    "outputId": "1f1f7d78-b8c4-447e-81f1-83bce6520935"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]']"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.convert_ids_to_tokens(new_ids)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yVOy3y3SPIo9",
    "outputId": "ba4c3cec-03ac-4817-e905-7ee6f99e79a2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[864, 5, 224, 190, 1]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.encode(new_sentence, add_special_tokens=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tGBDu8aoPIo9",
    "outputId": "910d92eb-e5bf-4f7d-f8b7-98f319f69f48"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[864,   5, 224, 190,   1]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer(new_sentence, add_special_tokens=False, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VB8-t1puPIo_",
    "outputId": "8c7e3c8f-ff02-438b-ef3c-1c341c465950"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': [3, 864, 5, 224, 190, 1, 2, 58, 48, 72, 32, 313, 46, 5, 1, 29, 2], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'follow the white rabbit neo'\n",
    "sentence2 = 'no one can be told what the matrix is'\n",
    "joined_sentences = tokenizer(sentence1, sentence2)\n",
    "joined_sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IjJ5EQqfPIo_",
    "outputId": "94615faa-0066-4466-b0b1-ab81cc738785"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', 'no', 'one', 'can', 'be', 'told', 'what', 'the', '[UNK]', 'is', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.convert_ids_to_tokens(joined_sentences['input_ids']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jNHKY9CrPIo_",
    "outputId": "d8057e1d-61ef-4a0e-d48c-8dc0a476da60"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': [[3, 864, 5, 224, 190, 1, 2, 0, 0, 0, 0], [3, 58, 48, 72, 32, 313, 46, 5, 1, 29, 2]], 'token_type_ids': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "separate_sentences = tokenizer([sentence1, sentence2], padding=True)\n",
    "separate_sentences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZOQDFqKbPIpF",
    "outputId": "08814631-8f28-4073-d2d1-fcdec4030b28"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', '[PAD]', '[PAD]', '[PAD]', '[PAD]']\n",
      "[1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.convert_ids_to_tokens(separate_sentences['input_ids'][0]))\n",
    "print(separate_sentences['attention_mask'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Uqsq7GS3PIpG",
    "outputId": "f248a47a-2b29-49db-839a-12f48fce19fe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'follow', 'the', 'white', 'rabbit', '[UNK]', '[SEP]', 'no', 'one', 'can', 'be', 'told', 'what', 'the', '[UNK]', 'is', '[SEP]']\n",
      "['[CLS]', 'another', 'first', 'sentence', '[SEP]', '[UNK]', 'second', 'sentence', 'here', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "first_sentences = [sentence1, 'another first sentence']\n",
    "second_sentences = [sentence2, 'a second sentence here']\n",
    "batch_of_pairs = tokenizer(first_sentences, second_sentences)\n",
    "first_input = tokenizer.convert_ids_to_tokens(batch_of_pairs['input_ids'][0])\n",
    "second_input = tokenizer.convert_ids_to_tokens(batch_of_pairs['input_ids'][1])\n",
    "print(first_input)\n",
    "print(second_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2U40RGXSPIpG",
    "outputId": "2c3207ec-e0fa-492f-a79c-5fe769e100cf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[   3,   27,    1,  ...,   99, 1085,    2],\n",
       "        [   3,   24,   11,  ..., 1302,    5,    2],\n",
       "        [   3,   49,   13,  ...,    0,    0,    0],\n",
       "        ...,\n",
       "        [   3,    1,    6,  ...,    0,    0,    0],\n",
       "        [   3,    6,  138,  ...,    0,    0,    0],\n",
       "        [   3,    1,    1,  ...,    0,    0,    0]])"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_dataset = tokenizer(dataset['sentence'],\n",
    "                              padding=True,\n",
    "                              return_tensors='pt',\n",
    "                              max_length=50,\n",
    "                              truncation=True)\n",
    "tokenized_dataset['input_ids']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RYIpul-cPIpG"
   },
   "source": [
    "# Before Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U80QzETEPIpH"
   },
   "source": [
    "## One-Hot Encoding (OHE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NS3mlNgIPIpH"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ohe1.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MLGxP_hdPIpH"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ohe2.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-K2WB4P5PIpH"
   },
   "source": [
    "## Bag of Words (BoW)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "L6ljfjKBPIpH",
    "outputId": "ec3a8665-6b73-4b36-f60a-e232b6fdf6f1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['the', 'white', 'rabbit', 'is', 'rabbit']"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'the white rabbit is a rabbit'\n",
    "bow_tokens = simple_preprocess(sentence)\n",
    "bow_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wLrldcH-PIpI",
    "outputId": "aa5dd700-a9d4-49fa-cf64-4e2c82af6fbe"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(42, 1), (79, 1), (246, 2), (248, 1)]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bow = dictionary.doc2bow(bow_tokens)\n",
    "bow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dfbvtiv2PIpI"
   },
   "source": [
    "## Language Models\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank1.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank2.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/fill1.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/fill2.png?raw=1)\n",
    "\n",
    "## N-grams\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/ngrams.png?raw=1)\n",
    "\n",
    "## Continuous Bag-of-Words (CBoW)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank_end.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/blank_center.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UxDgKYloPIpI"
   },
   "source": [
    "# Word Embeddings\n",
    "\n",
    "## Word2Vec\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/cbow.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "id": "atT-CpiCPIpI"
   },
   "outputs": [],
   "source": [
    "class CBOW(nn.Module):\n",
    "    def __init__(self, vocab_size, embedding_size):\n",
    "        super().__init__()\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_size)\n",
    "        self.linear = nn.Linear(embedding_size, vocab_size)\n",
    "\n",
    "    def forward(self, X):\n",
    "        embeddings = self.embedding(X)\n",
    "        bow = embeddings.mean(dim=1)\n",
    "        logits = self.linear(bow)\n",
    "        return logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xn-7a8LHPIpJ",
    "outputId": "2beac5af-7334-47ec-b721-1576fe5e8dd0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrderedDict([('weight',\n",
       "              tensor([[ 0.3367,  0.1288,  0.2345],\n",
       "                      [ 0.2303, -1.1229, -0.1863],\n",
       "                      [ 2.2082, -0.6380,  0.4617],\n",
       "                      [ 0.2674,  0.5349,  0.8094],\n",
       "                      [ 1.1103, -1.6898, -0.9890]]))])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(42)\n",
    "dummy_cbow = CBOW(vocab_size=5, embedding_size=3)\n",
    "dummy_cbow.embedding.state_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PavMubbxPIpJ"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VQ4ZKqOCPIpJ",
    "outputId": "4a8fee88-80ac-4ebd-c5a3-329e4a81c492"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 2.2082, -0.6380,  0.4617],\n",
       "        [ 0.2674,  0.5349,  0.8094]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# tokens: ['is', 'barking']\n",
    "dummy_cbow.embedding(torch.as_tensor([2, 3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "id": "Xeo7CI0FPIpK"
   },
   "outputs": [],
   "source": [
    "tiny_vocab = ['the', 'small', 'is', 'barking', 'dog']\n",
    "context_words = ['the', 'small', 'is', 'barking']\n",
    "target_words = ['dog']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "id": "64hplr5JPIpK"
   },
   "outputs": [],
   "source": [
    "batch_context = torch.as_tensor([[0, 1, 2, 3]]).long()\n",
    "batch_target = torch.as_tensor([4]).long()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uobj-IG8PIpK"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_cbow.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hMfauJsZPIpK",
    "outputId": "2f4ce22b-eb62-4b75-ac20-1d4848212826"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.7606, -0.2743,  0.3298]], grad_fn=<MeanBackward1>)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cbow_features = dummy_cbow.embedding(batch_context).mean(dim=1)\n",
    "cbow_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e19wHR2-PIpK"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/w2v_logits.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3e5bcIGOPIpL",
    "outputId": "df05479d-28ea-481f-be99-5bb67fbce5e5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.3542,  0.6937, -0.2028, -0.5873,  0.2099]],\n",
       "       grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logits = dummy_cbow.linear(cbow_features)\n",
    "logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BY_RwBJSPIpL"
   },
   "source": [
    "## What is an Embeddings Anyway?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9u0vTklVPIpL"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/rest_discrete.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QUBiuMFyPIpL"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/rest_continuous.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lPzPn8PaPIpM",
    "outputId": "83416a3b-b447-4478-95a8-9afc0c53895c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.0000, -0.4318,  0.9976, -0.2974],\n",
       "        [-0.4318,  1.0000, -0.4270,  0.3581],\n",
       "        [ 0.9976, -0.4270,  1.0000, -0.3598],\n",
       "        [-0.2974,  0.3581, -0.3598,  1.0000]])"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = torch.as_tensor([[.7, -.4, .7],\n",
    "                           [.3, .7, -.5],\n",
    "                           [.9, -.55, .8],\n",
    "                           [-.3, .8, .34]]).float()\n",
    "sims = torch.zeros(4, 4)\n",
    "for i in range(4):\n",
    "    for j in range(4):\n",
    "        sims[i, j] = F.cosine_similarity(ratings[i], ratings[j], dim=0)\n",
    "sims"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wkxlGEraPIpM"
   },
   "source": [
    "## Pre-trained Word2Vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Vrvd1NarPIpM"
   },
   "source": [
    "## Global Vectors (GloVe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wl_JFk35PIpM",
    "outputId": "45b37fab-2bdf-4aa9-a6d7-82383811adc8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "400000"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from gensim import downloader\n",
    "\n",
    "glove = downloader.load('glove-wiki-gigaword-50')\n",
    "\n",
    "# UPDATED\n",
    "###########################################################\n",
    "# The vocab property isn't available anymore at the newer\n",
    "# Gensim version. It was replaced by key_to_index\n",
    "# len(glove.vocab)\n",
    "len(glove.key_to_index)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fDzgsj0bPIpN",
    "outputId": "09675b72-d9ee-4c53-b18a-80dda0c4af2e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.16386 ,  0.57795 , -0.59197 , -0.32446 ,  0.29762 ,  0.85151 ,\n",
       "       -0.76695 , -0.20733 ,  0.21491 , -0.51587 , -0.17517 ,  0.94459 ,\n",
       "        0.12705 , -0.33031 ,  0.75951 ,  0.44449 ,  0.16553 , -0.19235 ,\n",
       "        0.065533, -0.12394 ,  0.61446 ,  0.89784 ,  0.17413 ,  0.41149 ,\n",
       "        1.191   , -0.39461 , -0.459   ,  0.022161, -0.50843 , -0.44464 ,\n",
       "        0.68721 , -0.7167  ,  0.20835 , -0.23437 ,  0.02604 , -0.47993 ,\n",
       "        0.31873 , -0.29135 ,  0.50273 , -0.55144 , -0.066692,  0.43873 ,\n",
       "       -0.24293 , -1.0247  ,  0.029375,  0.068499,  0.25451 , -1.9663  ,\n",
       "        0.26673 ,  0.88486 ], dtype=float32)"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove['alice']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "id": "46hbjBCmPIpO"
   },
   "outputs": [],
   "source": [
    "synthetic_queen = glove['king'] - glove['man'] + glove['woman']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 352
    },
    "id": "kcrptYwOPIpP",
    "outputId": "04333b76-b0b9-4fda-b824-5647e3d40dd5"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/content/plots/chapter11.py:23: UserWarning: FixedFormatter should only be used together with FixedLocator\n",
      "  axs[i].set_yticklabels(['', word, ''])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABvIAAAEuCAYAAABcVoUHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0pklEQVR4nO3debRddWE2/ufk3psQIMkNkJCRQaZoExuDVBlupIigSSQqDq8EhYhMxalYhtRKfwvfBWEo8lIsk9iXCFpfELSIJSgaCcGigogUBSpEkFngJiFkuLn3/P5IuSa9sSQB/H5JPp+1WIvss+/ez9lnz88ZGp2dnc0AAAAAAAAAVelXOgAAAAAAAADQlyIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAq1lg7A+nlwSXd6mmXm/fulK8vMeA1vH/BY6QhJkqXDxpWOkF88sbR0hOw1cmDpCEmSZqP8exHue3ZF6QiZsPI3pSOke8io0hGSJF1btJeOkCO/fnfpCLn8f72xdIQkSf+fXFM6QuaPObh0hCxatqp0hLxnm87SEZIkPQPbS0dIy9Lfl46QH6wYUTpC3rDdlqUjJEm2HdAoHSGp4Hyi37LO0hGSJP+5YqvSEfK7d767dITsd8aHS0eoRutbppWOkN/2DC4dIYefd2vpCEmSm0/7y9IRMvCJ/ygdITetHFM6QkYO2qJ0hCTJ8K3K39ZburKndITs0H956QhJksb9/146Qn6/+wGlI2TR8vLrxNV3P146QpLkpNxWOkL6vXV66QhptrSVjpAbH1pSOkKS5F2LFpSOkEe/c1PpCNnx6GNLR0iSNPuXuS5uNvpl2aANu5da/ojPeulpJt2Firyu8sffNJrdpSMkSRqN8jebVhVaD9ZUw3JIklSQo1TBvqYato/yr8RqNaybS1ZU8HpUsBySpLGq/BtBVlVwDFtZwY6i0axgQSRpxLJI6lgvm5XsuavYX1WQoXyC1SpYNbPquc7SEdLoquOGcA1q2G/XsL/qfKGrdIQkdewz+1Vw7VHDPYoKTu/+S/l1ooZttHyC1RrdNewryi+NCjbRLKvhhDtJI+Xf/F1+jUgV59s1HDuSOu6TNF94vnSEKs4nkqSngvsD66v8208BAAAAAACAPhR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhTa7Im/+/Plpb2/PmWee+ZLjTpgwIRMmTPgTpAIAAAAAAIC1bXZFHgAAAAAAALwWtJYOULN//dd/LR0BAAAAAACAzZQi73+w8847l44AAAAAAADAZspXa/6XRYsWZcqUKRk6dGguueSSJOv+jbwzzzwz7e3tmT9/fq6++urst99+GTFiRPbYY4+ccsopWbZsWZ9pr1q1Kuedd14mTpyY7bffPm9605ty3nnnZeHChWlvb8/xxx//J3mOAAAAAAAAvHb4RF6SJ554IoceemgeeOCBXH755Xnf+973kn9z2WWX5eabb86UKVMyefLk3Hzzzbnkkkvy7LPP5rLLLltr3BNOOCHf+MY3stNOO+XjH/94Vq5cmX/6p3/KT37yk1frKQEAAAAAAPAat9kXef/5n/+Z973vfXnuuefy//7f/8v++++/Xn83b968zJs3L7vttluSZNmyZeno6Mg3v/nNnH766Rk5cmSS5Ec/+lG+8Y1vZMKECZk7d2623HLLJMlnP/vZTJ48+VV5TgAAAAAAALz2bdZfrXnnnXfmne98Z1544YVcf/31613iJclxxx3XW+IlycCBA3PooYemp6cnd911V+/wb3zjG0mSk08+ubfES5IRI0bkuOOOe9nPAQAAAAAAgE3TZlvk/fjHP84hhxySrbbaKnPnzs3EiRM36O/XNf7o0aOTrP69vRfdc889SZK99967z/hvectbNmieAAAAAAAAbD422yLv7rvvzvPPP583vvGN2WmnnTb47wcNGtRnWEtLS5Kku7u7d9iSJUvSr1+/bLvttn3GHz58+AbPFwAAAAAAgM3DZlvkHX300fnIRz6S66+/Ph//+MezatWqV2U+gwYNSk9PT5555pk+jz311FOvyjwBAAAAAAB47dtsi7x+/frlggsuyBFHHJHrrrsuRx999KtS5o0fPz5J8u///u99HvvJT37yis8PAAAAAACATcNmW+QlSaPRyPnnn5+ZM2fmuuuuy1FHHfWKl3kf/OAHkyRnn312li1b1jv8ySefzMUXX/yKzgsAAAAAAIBNR2vpAKU1Go2cd9556devXy6//PI0m8185StfSWvrK7No9t9//3zgAx/I1VdfnX322SdTp07NihUr8q1vfSt77rlnbrzxxvTrt1n3qQAAAAAAAKzDZl/kJavLvHPPPTf9+vXLZZddlpkzZ+YrX/nKKzb9iy66KLvvvnuuvPLKXHrppRk1alSOP/74vO1tb8uNN96YQYMGvWLzAgAAAAAAYNOw2RV5HR0d6ezs7DO80WjknHPOyTnnnNM77Je//GWf8WbNmpVZs2atc9ozZszIjBkz+gxvbW3NSSedlJNOOmmt4XPmzEmS7LbbbhvyFAAAAAAAANgM+E7HP4Enn3wyzWZzrWGPPfZYzjnnnLS0tOTggw8ulAwAAAAAAIBabXafyCvhi1/8Ym666absvffeGTZsWH73u99l7ty5WbJkSU499dSMGTOmdEQAAAAAAAAqo8j7EzjwwANz33335aabbkpnZ2e22GKL/Nmf/VmOOuqofOADHygdDwAAAAAAgAop8v4EDjzwwBx44IGlYwAAAAAAAPAa4jfyAAAAAAAAoEI+kfca0a9Rbt5tFdS9zUZL6QhJkmazWTpCWguuCy+qYTkkSTPlc5TcNl9Uw/ZR/pVYrYZ1c9CACl6PCpZDkjRb+5eOkNYKjmH9K9hRNBsVLIgkzVgWSR3rZaOSPXcdu6vyIconWK2CVTOtQ9tLR0izbYvSEapRw367hv1V+5ZtpSMkqeMcr6eCa48a7lFUcHr3X8qvEzVso+UT/JeWGvYV5ZdGBZtoBtZwwp2kmQGlI1SwRtRx/Krh2JHUcZ+kseXWpSNUcT6RlLs/sDHzbXR2dpbfkgAAAAAAAIC1VNJFAwAAAAAAAGtS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVKi1dADWT2traxqNRpF5P9TZVWS+a3rdoNIJVmtZ/GTpCHnPNU+UjpBzP/jnpSMkSXZp6SwdIY/3G1o6QrbfsvyufHFXT+kISZLBbeXfn/Lciu7SETL86f8oHSFJ8rP+u5WOkFU9zdIR8pYBz5SOkGe2Gl06QpLkP55aWjpCJnffXzpCHhte/jg64sFbSkdIknzu0R1KR8jfHrBL6Qh5bvmq0hGSJBUcwnL5hP1LR8js+ReUjpD7h7yxdIQkSaFL0LXs0r/8sWOnQ88sHSFJsvDqk0tHSM8Wg0tHyJMryq+YXRWcY9ZiUP+W0hEysKX8OpEknSvLH0hHL/lN6QhZOXyP0hFy+6PPl46QJNlr1NalI+TJpeXv6y7sXF46QiZv8VTpCKv1K38Pb9X9Py0dIffsMq10hCTJ+GFbFJlvs9lM16oNuwYsv+awXhqNRrEir4bb8zVcQCZJo4KlsbiCGz3ll8JqjZS/eGqm/MpZat+wthoyWBa9CZrlLyCTpLv8JppVFWRoNGvYa5ZfL5NKXo+e8sfRKo4d3eUv6JNkaQ1vBKng2FHDOpEkPRWcWy195rnSEarYT1SwZSSp4+hRQ4ZnFpUvE5M6rn8q2GVWsc8s/0rUpPzrUcWKmTrWzRquPWq4Lq/huiOpY1nUsF7W8HrUsG0kSSrI0Vi1snSEKu4XJXVso+ur/EcXAAAAAAAAgD4UeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIU2myJv/vz5aW9vz5lnnpnbb78906ZNy5gxY7LLLrvks5/9bJYtW5YkmTt3bt7xjndk1KhR2W233XLaaadl1apVvdNZtGhRzj///EyZMiXjxo3LsGHDMm7cuBx77LF56KGH+sz3zDPPTHt7e+bPn5+rr746++23X0aMGJE99tgjp5xySu98AQAAAAAAYE2bTZH3ojvuuCPTp0/P4MGDc+SRR2bMmDG5/PLL8+lPfzrXXnttjjjiiIwdOzZHHnlkhgwZkgsuuCD/8A//0Pv3999/f84444wMHDgw06ZNy/HHH5+JEyfmmmuuyQEHHJCHH354nfO97LLL8pnPfCavf/3r87GPfSzt7e255JJL8qlPfepP9dQBAAAAAAB4DWktHeBP7fvf/36uuuqqTJ06NUnS1dWV/fffP1dffXVuvvnmfPe7382kSZOSJLNmzcqkSZNy8cUX58QTT0xbW1t233333HfffRk6dOha073lllvynve8J+eee24uuOCCPvOdN29e5s2bl9122y1JsmzZsnR0dOSb3/xmTj/99IwcOfJVfuYAAAAAAAC8lmx2n8jr6OjoLfGSpK2tLdOnT0+z2cw73/nO3hIvSQYNGpSDDz44zz33XB599NEkyZAhQ/qUeEkyefLkjBs3LvPmzVvnfI877rjeEi9JBg4cmEMPPTQ9PT256667XpknBwAAAAAAwCZjs/tE3oQJE/oMGzFixEs+9sQTT2SnnXZKsvr39i666KLccccdeeaZZ9b6Db3+/fuvc74TJ07sM2z06NFJVv/uHgAAAAAAAKxpsyvyBg0a1GdYS0vLSz7W1dWVJPnWt76VmTNnZuutt84BBxyQHXbYIQMHDkyj0cjXvva1PPLIIxs83+7u7o17MgAAAAAAAGyyNrsi7+WaPXt2tthii8ybNy+77LLLWo9de+21hVIBAAAAAACwqdnsfiPv5XrooYey++679ynxnnjiiSxcuLBMKAAAAAAAADY5irwNNHbs2Dz00EN56qmneoctX748J554Yu/XbwIAAAAAAMDL5as1N9AxxxyTk08+OZMnT84hhxyS7u7u/PCHP0yz2cz48eNzzz33lI4IAAAAAADAJsAn8jbQ0UcfnS9+8YsZOnRo5syZk+985zvZd999873vfS9DhgwpHQ8AAAAAAIBNxGbzibyOjo50dnau87EZM2ZkxowZ63xs1qxZmTVrVu+/G41GZs6cmZkzZ/YZ94YbbnjJv1/f+QIAAAAAALB584k8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqFBr6QCsn2azWWzeNbS9BZ/+WpoVLI3BW5TfbMsvhdWaaZSOkEbKr5wl9w9/UEMGy6I3QaOldIQkSUv5TTTNKjLUsNcsv14mSWsNr0e/8sfRKo4dLW2lIyRJtmqrYPuo4NhRwzqR1HGOt9W2Q0tHqGI/UcNrkaSCs+06jmDbDtmqdIQkdVz/VLDLrGKfWUOGelSwLCqIkNSxXtRw7VHDdXkN1x1JHcuihvWyhtejhm0jSVJBjmZr/9IRqrhflJTbRjdmvo3Ozs7yWzMAAAAAAACwlvIVMAAAAAAAANCHIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKKPAAAAAAAAKiQIg8AAAAAAAAqpMgDAAAAAACACinyAAAAAAAAoEKtpQOwftoaSaNRZt63Pb68zIzX8Lb+j5WOkCRptm5ROkI+cMNzpSNkRPvA0hGSJBd0bF06Qn6yfGjpCBm2ZVvpCNlxQPn9RJI83bNl6QgZ8cSdpSNk4bCJpSMkSZ5e2lU6Qh5ZVH7dfO/I8suh54Gflo6QJPnP1x1cOkLGDi6/z+xcsap0hIxa+tvSEZIk77zqqdIR8u1P7V06QhY8srh0hGrc/pdTS0fIqTfOLh0hC0f8RekISZIt28q/93dYd2fpCPnzE68vHSFJ8ov/877SEZJmT+kEWd5/SOkIeX5V+eWQJP37FbpRtGaGlvL7ie6eZukISZLfLyt/jrfjw/NLR0hz171KR8hvlpW/f5ckK3vK7yt2GtK/dITc90z56/I3PXlr6QhJkuYe+5SOkKVXX1g6Qh5+12dKR0iSjNtmQJH5NpvNdK3asGOGIu81olGwyFtVwflQo4KLhSRVXLQsXtFdOkK2Xlk+Q5I0Un7lrGH76En5i7fyCVZrVpCk0VP+4q2G5ZAk3RVsHysruKivYV/VWLWydIQkSfmjaNIodUK1hhq20VrOrRZVcMOrhnWiq46XowrLnin/pjXH8j+oIUcNx9Fnl5S/AZnUc85dXAX77WpejQqWRQ3H0Vpejir2md3l30RYg1pOrSq4HK1iP1HDvQHbxhpWvFA6QR3bRio5hq2n8m+bAQAAAAAAAPpQ5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFFHkAAAAAAABQIUUeAAAAAAAAVEiRBwAAAAAAABVS5AEAAAAAAECFihR5nZ2d2WabbfKhD31oreF333132tvb097engcffHCtx6ZOnZoRI0ZkxYoVvcOuvPLKvP3tb8/o0aMzevTovP3tb89VV13VZ37z589Pe3t7zjzzzNx+++2ZNm1axowZk1122SWf/exns2zZsiTJ3Llz8453vCOjRo3KbrvtltNOOy2rVq1aa1qLFi3K+eefnylTpmTcuHEZNmxYxo0bl2OPPTYPPfRQn3mfeeaZaW9vz/z583P11Vdnv/32y4gRI7LHHnvklFNO6Z03AAAAAAAArKlIkdfe3p7x48fnxz/+cbq7u3uHz58/f53/v3z58vzsZz/LXnvtlQEDBiRJTj755HziE5/I448/nsMPPzyHH354Hn/88Zxwwgk55ZRT1jnfO+64I9OnT8/gwYNz5JFHZsyYMbn88svz6U9/Otdee22OOOKIjB07NkceeWSGDBmSCy64IP/wD/+w1jTuv//+nHHGGRk4cGCmTZuW448/PhMnTsw111yTAw44IA8//PA6533ZZZflM5/5TF7/+tfnYx/7WNrb23PJJZfkU5/61EYvRwAAAAAAADZdraVm3NHRkbvvvjt33XVX9txzzySry7tdd901y5cvz/z583PEEUckSW6//fasWLEiHR0dSZIFCxbk0ksvzR577JGbbropQ4YMSZLMmjUrBx54YC655JJMnz49++yzz1rz/P73v5+rrroqU6dOTZJ0dXVl//33z9VXX52bb7453/3udzNp0qTeaU2aNCkXX3xxTjzxxLS1tSVJdt9999x3330ZOnToWtO+5ZZb8p73vCfnnntuLrjggj7Pd968eZk3b1522223JMmyZcvS0dGRb37zmzn99NMzcuTIV2S5AgAAAAAAsGko9ht5L5Zyt9xyS5Kku7s7t912Wzo6OrLffvut89N5++23X5Lk61//epLk1FNP7S3xktWf9Hvx03hf+9rX1jnPF0u8JGlra8v06dPTbDbzzne+s7fES5JBgwbl4IMPznPPPZdHH320d/iQIUP6lHhJMnny5IwbNy7z5s1b5/M97rjjeku8JBk4cGAOPfTQ9PT05K677lrn3wAAAAAAALD5Klbk7bPPPmlpaekt6e6+++4sXrw4kydPTkdHR5588sncd999SZJbb701AwcOzJvf/ObecZM/FHtrerEg/OUvf9nnsQkTJvQZNmLEiJd87Iknnlhr+Pz583PYYYdljz32yHbbbdf7u3733ntvn3FfNHHixD7DRo8enWT17+4BAAAAAADAmop9tebgwYPz53/+57n99tvT1dWV+fPnp9FopKOjIy+88EKS1YXZ2LFjc8cdd2TfffdN//79kyRLlixJv379st122/WZ7vDhw9NoNLJkyZI+jw0aNKjPsJaWlpd8rKurq3fYt771rcycOTNbb711DjjggOywww4ZOHBgGo1Gvva1r+WRRx5Z5/P9n6a/5u8EAgAAAAAAQFKwyEtWf3ruzjvvzB133JFbb701r3/963vLuR133DHz58/PLrvskq6urt5P2iWrS7Genp78/ve/z7Bhw9aa5tNPP51ms7nO4uyVMHv27GyxxRaZN29edtlll7Ueu/baa1+VeQIAAAAAALD5KfbVmskfvgbzBz/4QX784x+vVdZNnjw5t956a+9v6K35NZpvfOMbk6z+ys3/7sVh6/qqzFfCQw89lN13371PiffEE09k4cKFr8o8AQAAAAAA2PwULfLe+ta3prW1NV/5yleyZMmSTJ48ufexjo6OPPPMM/nqV7+arbbaKpMmTep97MMf/nCS5KyzzsrixYt7hy9atChnnXXWWuO80saOHZuHHnooTz31VO+w5cuX58QTT1zrKzgBAAAAAADg5Sj61Zpbb711Jk2alJ/85Cfp169f9t13397HXvx03u9///u8/e1vT1tbW+9j++67b4455phceuml2WefffLud787zWYz119/fR599NEce+yxa03rlXTMMcfk5JNPzuTJk3PIIYeku7s7P/zhD9NsNjN+/Pjcc889r8p8AQAAAAAA2LwU/URe8ofC7o1vfGPa29t7h48cOTK77rprkrW/VvNFZ599di688MIMHz48V1xxRebMmZPtt98+F154Ye+n8l4NRx99dL74xS9m6NChmTNnTr7zne9k3333zfe+970MGTLkVZsvAAAAAAAAm5dGZ2dns3QIXlr/fkmjUWbeP3p0eZkZr+HA/r8rHSFJ0mzdonSEHHTds6UjZNTQgaUjJEn++e2DS0fIgmXblI6Q7bfqXzpCXjdgWekISZIne7YqHSGjHru9dIQ8uP1epSMkSZ58fmXpCFnYWX7d/F+jy3/1ds+vbisdIUly3+7vLh0hOw0pv898dvmq0hEy5vkHS0dIkrz18idKR8iPTp380iO9yn64cFHpCNVYMHGf0hFy+u1fKh0hD456db5hZkNt2Vb8vb/Zvrv89c+Yo68pHSFJ8ruvHFY6QtLsKZ0gywa0l46Q57vKL4ck6d9S6EbRGga0lN9PrOqp49bm0y+UP8fb+aGbS0dIc4/yx/L7l5W/f5ckK7vL7yteN3RA6Qi59+ny1+V7Pf7D0hGSJM03vK10hDx/5dmlI2Th9L8tHSFJ8mfbldlXNJvNrNzAn2krf7QFAAAAAAAA+lDkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUqLV0ANZPs+Dv9raW/+3kNBt1dM415Bg8oKV0hGzdv3yGJGmm/MpZw/bRL+V/2Lt8gtUaFSRp9it/aK1hOSRJSwXbR/9+5UPUsK9qtvYvHSFJHe8ga5Y8qfovNWyjNZzTJMmQgeX3mTWsE211vBxVGLjt0NIRHMvXUEOOGo6j2wzaonSEJPWccxdXwX67mlejghg1HEfrWCcq2We2tJWOUMFSqOO6I0kquBytYvuo4d5ADdtGUsf2kQFblk5Qx7aRcsewjZlvo7Ozs4r1BwAAAAAAAPiDWt6gAAAAAAAAAKxBkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUCFFHgAAAAAAAFRIkQcAAAAAAAAVUuQBAAAAAABAhRR5AAAAAAAAUKHW0gFYP8/3NJI0isx7u+7FRea7lod+XjpBkuTmrfcsHSEH9XuwdITc2P260hGSJNsMbCsdIVv9f8eVjpCc8eXSCfLRM24sHSFJstP4saUj5P98YELpCBnRv7t0hCTJqpYBpSOkJT2lI+Se368sHSGtLWXOIf67nz26qHSEHD5heOkI6ffL75WOkOw8sXSCJEljVfntY/7zg0tHyOTu+0tHSJJ0P/1o6QhZtechpSNk8cryx9Er73q8dIQkyZ6jh5SOkI5ld5eOkKk3tZSOkCS5fp/nSkdI18R3lY6Q1p6u0hFyx9PlMyRJ+xblr4lHbV3+1uLn59ZxHD3jXXuUjpBHFpdfN7dsK3/tMXb5I6UjJElu79q+dIR0Li+/Tjy7rHyG/zVieekISZKbFg0qHSFvGrl16QgZ9kId57r9lj5bZL7NRkteGL77Bv1N+aMt66lckddIs8h819JdfoefJF3l7wenkfI3FmpYDknSXcGq2bNkSekIqWETfe758jdik2SbFeW3j2ahffWaGuUjJEkaFQRpVPB61LCv6ldBhiRZUcHCqGK9rOK8pvxrkdRxnrmqfIQ0elaVjpCkjmK1hm201HXXmpbVsGLG9vGiRcsqWBBJGt220dUZSieoY9tIUsFb1lLFC/L8yiqWRBXbRw1Loopr4mYNS6KOfUUN9/CquA6sYuuo4/Wo4Vy3ltej0Sx/H3F9+WpNAAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACq0yRV5U6dOTXt7+2Y7fwAAAAAAADYNr7ki76qrrkp7e3uuuuqqIvM/88wz097envnz5xeZPwAAAAAAAJuH1tIBNjUXX3xxli1bVjoGAAAAAAAAr3GKvFfY2LFjS0cAAAAAAABgE7DBX6357W9/O1OmTMmuu+6a7bffPuPGjcv06dPz7W9/O7/5zW8ydOjQfOADH1jn3y5ZsiSjR4/OXnvt1Tvs+OOPT3t7exYuXJiLL744e+21V4YPH57x48dn9uzZ6enpWWvcE044IUlywgknpL29vfe//66rqytnnnlmJkyYkOHDh2fPPffMl7/85XXmajab+epXv5qDDz44Y8eOzciRI7P//vvnq1/96lrjTZ06NWeddVaS5N3vfnfvvCdMmLDWOH/sN/JuuOGGvPe9783OO++c7bffPhMmTMgxxxyTe++9d53jAwAAAAAAsPnaoE/kXX755fnsZz+bESNGZNq0adlmm23y5JNP5s4778wNN9yQ6dOnp6OjIzfffHN+97vfZcyYMWv9/TXXXJOlS5fmox/9aJ9pn3baaVmwYEEOPvjgHHDAAbnhhhsye/bsdHV15fOf/3yS1SXZokWL8t3vfjdTpkxZq0D774466qjceeedOfDAA9PS0pLrrrsuf/M3f5O2trYcccQRveM1m80cffTRueaaa7LLLrvk/e9/f9ra2jJv3rx88pOfzH333Zf//b//d5LksMMOS5IsWLAgH/7wh7PDDjskSYYMGfKSy+5zn/tcvvSlL2Xo0KGZOnVqhg0blkcffTQ/+tGPMnHixLzhDW94yWkAAAAAAACw+digIm/OnDnp379/5s+fn2HDhq312LPPPpskmTlzZm655ZZceeWVOfXUU9f59x/+8If7TPsXv/hFFixYkBEjRiRJTj755EyaNCmXXnppTjnllPTv3z/Tpk3rLfKmTp2aGTNm/NGsjz32WG677bYMHjw4SXLcccdl7733zoUXXrhWkTdnzpxcc801mTFjRs4///y0tbUlSVauXJmPfvSjufDCC/P+978/EydOzIwZM/Lwww9nwYIFOeyww9LR0bFey+3GG2/Ml770pbzhDW/Id77znWyzzTa9j61atap32QEAAAAAAMCLNvirNdva2nrLrjW9WE5NmzYtw4cPz1VXXbXW12Lec889+fnPf54pU6Zku+226/P3J510Um+JlyTbbrttpkyZkiVLluSBBx7Y0Jg57bTTeku8JNltt93ylre8JQ888ECWLFnSO/zSSy/NVlttlXPPPXet59W/f//eTwJec801Gzz/NV1++eVJktmzZ69V4iVJa2trhg8f/rKmDwAAAAAAwKZngz6Rd+ihh+a0007L3nvvnfe///3p6OjIW9/61rUKs7a2tsyYMSNf/OIX84Mf/CAHHnhgkuSKK65IkrU+DbemiRMn9hk2evToJMmiRYs2JOZ6TW/QoEF54YUXcu+992bkyJE5//zz+4y/atWqJNmoInFNd9xxRwYMGJD99tvvZU0HAAAAAACAzccGFXmf/OQnM3To0HzlK1/JhRdemH/8x39Ma2trDjrooJxxxhnZaaedkqwu684///zMmTMnBx54YJYvX56rr746O+64Y/bff/91TnvQoEF9hrW0tCRJuru7N+xZJWuVi39sep2dnWk2m3nsscdy1lln/dFpLV26dIPnv6bFixdn5MiR6ddvgz8ACQAAAAAAwGZqg4q8RqORj3zkI/nIRz6SZ599Nrfddlu++c1v5rrrrsuDDz6YBQsWpKWlJTvttFMOOOCA/Nu//Vuefvrp/PCHP0xnZ2c+8YlPpNFovFrPZYO9WB5OnDgx8+bNe9XmM2TIkDz11FPp6elR5gEAAAAAALBeNrpV2mabbTJt2rT88z//cyZPnpxf//rXefDBB3sfP/LII9PV1ZWvf/3rmTNnTlpaWjJjxoyXHfjlfErvvxs0aFD22GOP3H///ens7Nyg+a/5+38vZc8998yKFSty6623bkxMAAAAAAAANkMbVOTNnz8/zWZzrWFdXV157rnnkiQDBgzoHf6ud70rI0eOzD/90z9lwYIFOeiggzJy5MiXHXjo0KFJkkcfffRlTytJjj322Lzwwgv59Kc/vc6v0Fy4cGF++9vf9pn/7373u/Wex8c//vEkyamnntq7rF60atWqPPXUUxsTHQAAAAAAgE3YBn215owZMzJ48OC8+c1vztixY9PV1ZV58+bl17/+daZPn54ddtjhDxNubc3hhx+ec845J8nq3817JfzFX/xFBg4cmIsuuiidnZ3ZbrvtkiQnnXTSRk1v5syZ+elPf5qvf/3ruf322/O2t70tI0eOzFNPPZUHHnggP/vZz/LlL385O+64Y5Kko6MjjUYjX/jCF/LrX/86gwcPzpAhQ3LMMcf80XkcdNBB+eQnP5l//Md/zKRJkzJt2rQMGzYsjz32WG655ZZ84hOfyF/91V9tVH4AAAAAAAA2TRtU5P393/99vv/97+eOO+7IjTfemC233DI777xzzjvvvHzkIx/pM/5hhx2Wc845J6NGjco73vGOVyTw0KFDc8UVV2T27NmZM2dOli1blmTji7xGo5GLLrooBx10UK644orMnTs3S5cuzbBhw/K6170uX/jCF7L//vv3jj9u3Lh86UtfyoUXXphLL700K1asyNixY//HIi9JvvCFL2SvvfbKZZddlm9/+9tZsWJFtt9++3R0dOQv//IvNyo7AAAAAAAAm64NKvKOOuqoHHXUUes9/t13351k9Sf5Xvxtuf/uoosuykUXXbTOx2bNmpVZs2b1GX7QQQfloIMOWuff3HDDDX80z/80r/e+971573vf+0f/dk2HHXZYDjvssA2e/yGHHJJDDjlkveYBAAAAAADA5m2DfiNvQzSbzVx44YVpbW19xb5WEwAAAAAAADYXG/SJvPXxH//xH5k7d25uv/32/PSnP83MmTMzZsyYV3o2AAAAAAAAsEl7xYu8u+66K6effnoGDx6cD33oQ/nCF77wSs8CAAAAAAAANnmveJE3Y8aMzJgx45WeLAAAAAAAAGxWXrXfyAMAAAAAAAA23iv+iTxeLc2Cc24Um3evlrbSCZIkbRVU381+LaUjpK3c6riWlgpWzX6DBpWOkBo20aFb9y8dIUkyeED57aNRcH/9omb5CEmSZgVBmhW8HlXsqyrIkCQDKlgYVayXVZzXlH8tViufo7V8hDT71XFZ1mwtfzyvYRstee31ooE1rJixfbxoyMDy55hJ0myxja7OUDpBHdtGUsm78yt4QbbuX8WSqGL7qGFJVHFN3KhhSdSxr6jhXmYV14FVbB11vB41nOvW8no0G2XO8TZmvo3Ozs7yrxwAAAAAAACwljqqTwAAAAAAAGAtijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAop8gAAAAAAAKBCijwAAAAAAACokCIPAAAAAAAAKqTIAwAAAAAAgAq1lg7A+nlyWdJTaN7Hffknheb8B9eN+XHpCEmSLT74N6Uj5OHFXaUj5OkXymdIkj2HtZWOkH/9zZLSETJ1t21KR0j//1xQOkKS5PExbykdIf/2wDOlI+QN221dOkKS5C3tK0tHyO96yi+LJ5eW32e+efHPS0dIkizaae/SETL0uQdKR8jjg3YtHSGPLF5ROkKSZJuB5Y/lOw5qKR0hj79Q6ky/PktPPqp0hOzyoYNKR0i/fd5fOkKSpPW5h0tHyLLtdisdIYfPqeM4euZ7xpeOkNdtsbx0hDy8covSEbJsVbN0hCTJdluWP4Y9VsE9itdvV36dSJK2e39QOkK6n3m8dIQ8MOEDpSNkyIA6Pruy/YDy+4rWhXeUjpDuHd9UOkKe7Cq/v0yS0c8/VDpC/u3gvyodIfv99KbSEZIkg35/X5H5Nhv98sLQnTfobxR5rxE9KVfkLVq2qtCc17BiWekESZJGo1E6QnpSPkMl1yyp4OXIyp7yC6OG9bLRU8F+Ikmzgu1jRXf5daKabTTlg9SwTlSwSlSzjdaw4240yxcm1ss/KP9q1HEcrWGdqEXP8+XfJNXoKl90V7BaJqljn1nDNrp4RXfpCEkq2WeWDpA69pk9zUoOpBUsixrOKWrYTyRJo7t8qVnDMayC2yRV7CeSpNEovzBquBasYROtZp2o4Nyq67nO0hHqWCmSNJp1nOOtjzrengAAAAAAAACsRZEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABUSJEHAAAAAAAAFVLkAQAAAAAAQIUUeQAAAAAAAFAhRR4AAAAAAABU6DVR5HV3d+f888/Pm970pmy//fZ505velPPOOy8LFy5Me3t7jj/++N5x29vbM3Xq1HVOZ8KECZkwYUKf4StXrsyFF16YyZMnZ9SoURkzZkze9a535bvf/e46p7Mh4x9//PFpb2/PwoULc/HFF2evvfbK8OHDM378+MyePTs9PT0buVQAAAAAAADYlLWWDrA+Pv3pT+fKK6/MjjvumI9//ONZsWJFvvSlL+X2229/2dNesWJFDj300Nx6662ZMGFCDj/88KxatSo33XRTDjvssJx99tk55phjNnr8F5122mlZsGBBDj744BxwwAG54YYbMnv27HR1deXzn//8y34eAAAAAAAAbFqqL/Lmz5+fK6+8MuPHj8/cuXOz1VZbJUlOPPHEdHR0vOzpn3322bn11ltz0kkn5W//9m/TaDSSJEuWLMkhhxySv/u7v8u73/3ujBw5cqPGf9EvfvGLLFiwICNGjEiSnHzyyZk0aVIuvfTSnHLKKenfv//Lfi4AAAAAAABsOqr/as1/+Zd/SbK6+HqxxEuSUaNG5bjjjntZ0+7p6cnll1+enXfeea1SLkkGDRqUk08+OStXrsz111+/UeOv6aSTTuot8ZJk2223zZQpU7JkyZI88MADL+t5AAAAAAAAsOmp/hN599xzT5Jkn3326fPY3nvv/bKm/cADD6SzszMjR47M7Nmz+zz+zDPP9I63MeOvaeLEiX2GjR49OkmyaNGijX4OAAAAAAAAbJqqL/IWL16cfv36Zdttt+3z2PDhw1/WtJ977rkkya9+9av86le/+qPjLV26dKPGX9OgQYP6DGtpaUmSdHd3r39oAAAAAAAANgvVF3mDBw9OT09PnnnmmWy33XZrPfbUU0/1Gb/RaPzRYmzx4sUZPHhw779fLNcOOeSQzJkz5yWzbOj4AAAAAAAAsLGq/4288ePHJ0luu+22Po/9+Mc/7jOsvb09jz32WJ/hv/3tb/t8heUee+yRwYMH5+c//3m6urpeMsuGjg8AAAAAAAAbq/oi70Mf+lCS5Oyzz17rKysfe+yxXHzxxX3GnzRpUh5++OHceuutvcNWrlyZz33uc33GbW1tzcc+9rE88sgj+bu/+7t1lnP33ntvnn766Y0aHwAAAAAAADZW9V+tOXny5MyYMSNXXXVV9tlnn0ybNi0rV67Mtddemze/+c2ZO3fuWuOfcMIJ+cEPfpAPfvCDOfTQQzNw4MDMmzcvQ4YMyYgRI/pMf9asWfnFL36RSy65JDfddFP22WefDBs2LI899ljuvffe3HPPPfne976XYcOGbdT4AAAAAAAAsDGqL/KS5IILLsiuu+6aK664IpdddllGjRqVE044Ie9973v7FHkHHHBA/u///b8566yz8o1vfCNDhw7N9OnTc9ppp2XvvffuM+0BAwbkmmuuyVe/+tX8y7/8S66//vqsWLEiw4YNy7hx4/Kxj30sb3jDGzZ6fAAAAAAAANgYr4kir6WlJX/913+dv/7rv15r+G9/+9t1jj99+vRMnz69z/Bf/vKXf3T6Rx55ZI488sj1zrO+41900UW56KKL1vnYrFmzMmvWrPWaJwAAAAAAAJuX6n8jDwAAAAAAADZHijwAAAAAAACokCIPAAAAAAAAKvSa+I28P2bHHXdMZ2dn6RgAAAAAAADwivOJPAAAAAAAAKjQa/oTeZuTko3rkIEVrCYDBpZOkCRpNpulI6RfymdobZROsFoFL0f69yu/MGpYL5v9KthPJGlUsH0MaCm/TlSzjaZ8kBrWiQpWiWq20Rp23M1G+fexWS//oPyrUcdxtIZ1ohb9th5UOkKabQNKR6hhd5mkjn1mDdvo4AEtpSMkqWSfWTpA6thnVnAZ+F/KL4sazilq2E8kSbOlrXSEKo5hNWwfNewnkjqO5zVcC9awHKpZJyo4t2ob2l46Qh0rRZJmo8w53sasB43Ozs46lhoAAAAAAADQq3wFDAAAAAAAAPShyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQoo8AAAAAAAAqJAiDwAAAAAAACqkyAMAAAAAAIAKKfIAAAAAAACgQv8/8Zish2uD/N0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1800x350 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plot_word_vectors(glove,\n",
    "                        ['king', 'man', 'woman', 'synthetic', 'queen'],\n",
    "                        other={'synthetic': synthetic_queen})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gtLfGWX7PIpP",
    "outputId": "3ba8afb9-a739-49a5-eb3c-a2c07229d61c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('king', 0.8859834671020508),\n",
       " ('queen', 0.8609582185745239),\n",
       " ('daughter', 0.7684512734413147),\n",
       " ('prince', 0.7640699744224548),\n",
       " ('throne', 0.7634970545768738)]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove.similar_by_vector(synthetic_queen, topn=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fEk297ScPIpQ"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/embed_arithmetic.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6uv9IVvGPIpQ"
   },
   "source": [
    "$$\n",
    "\\Large\n",
    "w_{\\text{king}} - w_{\\text{man}}\\approx w_{\\text{queen}}-w_{\\text{woman}} \\implies w_{\\text{king}} - w_{\\text{man}} + w_{\\text{woman}} \\approx w_{\\text{queen}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O7M0z1YMPIpQ"
   },
   "source": [
    "## Using Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MuAY6Gk6PIpQ"
   },
   "source": [
    "### Vocabulary Coverage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bpLS4nhMPIpQ",
    "outputId": "8d9ca8fb-0816-413a-ed05-1ceabe47bf71"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3695"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab = list(dictionary.token2id.keys())\n",
    "len(vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AWvfyewTPIpQ",
    "outputId": "df5d4a4b-907e-4051-be67-63255c001f70"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42\n",
      "['[PAD]', '[UNK]', 'barrowful', 'beauti', 'carecrow']\n"
     ]
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# The vocab property isn't available anymore at the newer\n",
    "# Gensim version. It was replaced by key_to_index\n",
    "# unknown_words = sorted(list(set(vocab).difference(set(glove.vocab))))\n",
    "\n",
    "unknown_words = sorted(list(set(vocab).difference(set(glove.key_to_index))))\n",
    "###########################################################\n",
    "print(len(unknown_words))\n",
    "print(unknown_words[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "EddC0V1zPIpR",
    "outputId": "b7119697-844c-40bd-a731-265f3b54a307"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(80, 50539)"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unknown_ids = [dictionary.token2id[w] for w in unknown_words if w not in ['[PAD]', '[UNK]']]\n",
    "unknown_count = np.sum([dictionary.cfs[idx] for idx in unknown_ids])\n",
    "unknown_count, dictionary.num_pos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "id": "IWUf3u6CPIpR"
   },
   "outputs": [],
   "source": [
    "def vocab_coverage(gensim_dict, pretrained_wv, special_tokens=('[PAD]', '[UNK]')):\n",
    "    vocab = list(gensim_dict.token2id.keys())\n",
    "    # UPDATED\n",
    "    ###########################################################\n",
    "    # The vocab property isn't available anymore at the newer\n",
    "    # Gensim version. It was replaced by key_to_index\n",
    "    # unknown_words = sorted(list(set(vocab).difference(set(pretrained_wv.vocab))))\n",
    "    unknown_words = sorted(list(set(vocab).difference(set(pretrained_wv.key_to_index))))\n",
    "    ###########################################################\n",
    "    unknown_ids = [gensim_dict.token2id[w] for w in unknown_words if w not in special_tokens]\n",
    "    unknown_count = np.sum([gensim_dict.cfs[idx] for idx in unknown_ids])\n",
    "    cov = 1 - unknown_count / gensim_dict.num_pos\n",
    "    return cov"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eYZTCStNPIpR",
    "outputId": "1df5d6a6-7c10-461b-dab7-57ffa9b8a7e4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9984170640495459"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab_coverage(dictionary, glove)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EKzOosVxPIpR"
   },
   "source": [
    "### Tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "id": "Mf8fyy3mPIpS"
   },
   "outputs": [],
   "source": [
    "def make_vocab_from_wv(wv, folder=None, special_tokens=None):\n",
    "    if folder is not None:\n",
    "        if not os.path.exists(folder):\n",
    "            os.mkdir(folder)\n",
    "\n",
    "    # UPDATED\n",
    "    ###########################################################\n",
    "    # The index2word property isn't available anymore at the \n",
    "    # newer Gensim version. It was replaced by index_to_key\n",
    "    # words = wv.index2word\n",
    "    words = wv.index_to_key\n",
    "    ###########################################################\n",
    "    if special_tokens is not None:\n",
    "        to_add = []\n",
    "        for special_token in special_tokens:\n",
    "            if special_token not in words:\n",
    "                to_add.append(special_token)\n",
    "        words = to_add + words\n",
    "\n",
    "    with open(os.path.join(folder, 'vocab.txt'), 'w') as f:\n",
    "        for word in words:\n",
    "            f.write(f'{word}\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "id": "K4Y-ub7iPIpS"
   },
   "outputs": [],
   "source": [
    "make_vocab_from_wv(glove, 'glove_vocab/', special_tokens=['[PAD]', '[UNK]'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "id": "3NIjvZpFPIpS"
   },
   "outputs": [],
   "source": [
    "glove_tokenizer = BertTokenizer('glove_vocab/vocab.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IN_vm748PIpS",
    "outputId": "e0286629-ceab-4f08-b22a-af1b21b55579"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7101, 930, 2, 300, 12427]"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove_tokenizer.encode('alice followed the white rabbit', add_special_tokens=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "TwsVqJhtPIpS",
    "outputId": "0985270c-e9d5-478a-b5e4-4f567bacc33b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400002, 400000)"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(glove_tokenizer.vocab), len(glove.vectors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YBrqOhP-PIpT"
   },
   "source": [
    "### Special Tokens' Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "id": "UzAqTQ9CPIpT"
   },
   "outputs": [],
   "source": [
    "special_embeddings = np.zeros((2, glove.vector_size))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4AWIyA3JPIpT",
    "outputId": "b437d71a-a4c8-491e-a7f9-cf6b06bd060e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(400002, 50)"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extended_embeddings = np.concatenate([special_embeddings, glove.vectors], axis=0)\n",
    "extended_embeddings.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fMTUfdESPIpT",
    "outputId": "0b19685f-8abf-4b49-aae9-47c967b29dc2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alice_idx = glove_tokenizer.encode('alice', add_special_tokens=False)\n",
    "np.all(extended_embeddings[alice_idx] == glove['alice'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k8_PGFaXPIpU"
   },
   "source": [
    "## Model I - GloVe + Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rzpeWYcdPIpU"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "id": "QOc1ZhgdPIpU"
   },
   "outputs": [],
   "source": [
    "train_sentences = train_dataset['sentence']\n",
    "train_labels = train_dataset['labels']\n",
    "\n",
    "test_sentences = test_dataset['sentence']\n",
    "test_labels = test_dataset['labels']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "id": "r5SLBMioPIpU"
   },
   "outputs": [],
   "source": [
    "train_ids = glove_tokenizer(train_sentences,\n",
    "                            truncation=True,\n",
    "                            padding=True,\n",
    "                            max_length=60,\n",
    "                            add_special_tokens=False,\n",
    "                            return_tensors='pt')['input_ids']\n",
    "train_labels = torch.as_tensor(train_labels).float().view(-1, 1)\n",
    "\n",
    "test_ids = glove_tokenizer(test_sentences,\n",
    "                           truncation=True,\n",
    "                           padding=True,\n",
    "                           max_length=60,\n",
    "                           add_special_tokens=False,\n",
    "                           return_tensors='pt')['input_ids']\n",
    "test_labels = torch.as_tensor(test_labels).float().view(-1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "id": "3Gy9RIDEPIpU"
   },
   "outputs": [],
   "source": [
    "train_tensor_dataset = TensorDataset(train_ids, train_labels)\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_tensor_dataset, batch_size=32, shuffle=True, generator=generator)\n",
    "test_tensor_dataset = TensorDataset(test_ids, test_labels)\n",
    "test_loader = DataLoader(test_tensor_dataset, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j-3CdOpWPIpU"
   },
   "source": [
    "### Pre-Trained PyTorch Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "id": "bGxrKlyDPIpV"
   },
   "outputs": [],
   "source": [
    "extended_embeddings = torch.as_tensor(extended_embeddings).float()\n",
    "torch_embeddings = nn.Embedding.from_pretrained(extended_embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SpdJmQNsPIpV",
    "outputId": "904dd913-7461-4f2e-d2d9-58ec024d0214"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[   63,     2, 16315,  ...,     0,     0,     0],\n",
       "        [  102,    20, 22120,  ...,     0,     0,     0],\n",
       "        [  199, 26230,    66,  ...,     0,     0,     0],\n",
       "        ...,\n",
       "        [   73, 10580,    37,  ...,     0,     0,     0],\n",
       "        [    2, 12427, 24150,  ...,     0,     0,     0],\n",
       "        [ 2319,     3,    21,  ...,     0,     0,     0]])"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_ids, labels = next(iter(train_loader))\n",
    "token_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rwyGjFNSPIpV",
    "outputId": "711a6f73-1fe1-4213-f832-1b62257911a3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([32, 60, 50])"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings = torch_embeddings(token_ids)\n",
    "token_embeddings.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UIHGuQ4WPIpV",
    "outputId": "dd7ad341-fb98-4e26-ab39-6d631fdd2116"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 4.7639e-02,  5.8086e-02, -8.9823e-02,  ..., -2.9921e-01,\n",
       "          5.4377e-02, -1.8024e-01],\n",
       "        [ 5.5755e-02, -1.2622e-02, -3.7579e-02,  ..., -1.6176e-01,\n",
       "          3.5995e-02, -1.2326e-01],\n",
       "        [ 5.7766e-02,  6.9307e-03, -2.4787e-04,  ...,  3.0591e-02,\n",
       "         -9.5373e-03,  1.7501e-02],\n",
       "        ...,\n",
       "        [ 1.9234e-01,  2.3260e-01, -1.1317e-01,  ..., -2.4776e-01,\n",
       "         -5.3054e-02,  5.9994e-02],\n",
       "        [ 3.1925e-02, -1.1354e-02, -4.3769e-02,  ..., -3.6514e-02,\n",
       "         -2.9422e-03, -1.3578e-02],\n",
       "        [ 2.4775e-01,  1.6112e-01, -3.8034e-02,  ..., -1.7135e-01,\n",
       "         -3.1186e-02,  1.2527e-02]])"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings.mean(dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PT2eZ1EqPIpV",
    "outputId": "d3e5d701-3323-4d4f-8847-28a71b299476"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 4.7639e-02,  5.8086e-02, -8.9823e-02,  ..., -2.9921e-01,\n",
       "          5.4377e-02, -1.8024e-01],\n",
       "        [ 5.5755e-02, -1.2622e-02, -3.7579e-02,  ..., -1.6176e-01,\n",
       "          3.5995e-02, -1.2326e-01],\n",
       "        [ 5.7766e-02,  6.9307e-03, -2.4787e-04,  ...,  3.0591e-02,\n",
       "         -9.5373e-03,  1.7501e-02],\n",
       "        ...,\n",
       "        [ 1.9234e-01,  2.3260e-01, -1.1317e-01,  ..., -2.4776e-01,\n",
       "         -5.3054e-02,  5.9994e-02],\n",
       "        [ 3.1925e-02, -1.1354e-02, -4.3769e-02,  ..., -3.6514e-02,\n",
       "         -2.9422e-03, -1.3578e-02],\n",
       "        [ 2.4775e-01,  1.6112e-01, -3.8034e-02,  ..., -1.7135e-01,\n",
       "         -3.1186e-02,  1.2527e-02]])"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "boe_mean = nn.EmbeddingBag.from_pretrained(extended_embeddings, mode='mean')\n",
    "boe_mean(token_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UDyYigyhPIpV"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "id": "3yaB00G4PIpW"
   },
   "outputs": [],
   "source": [
    "extended_embeddings = torch.as_tensor(extended_embeddings).float()\n",
    "boe_mean = nn.EmbeddingBag.from_pretrained(\n",
    "    extended_embeddings, mode='mean'\n",
    ")\n",
    "torch.manual_seed(41)\n",
    "model = nn.Sequential(\n",
    "    # Embeddings\n",
    "    boe_mean,\n",
    "    # Classifier\n",
    "    nn.Linear(boe_mean.embedding_dim, 128),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(128, 1)\n",
    ")\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "id": "KTWpbPdfPIpW"
   },
   "outputs": [],
   "source": [
    "sbs_emb = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_emb.set_loaders(train_loader, test_loader)\n",
    "sbs_emb.train(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "qiaVP9uvPIpW",
    "outputId": "a2f11e81-552d-4baa-d723-d38ccc6a5b40"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACjUklEQVR4nOzdd3yN1wMG8Oe9O1MICULEKKlZ2qpNzRqxxYhRVaulNq1Ss2iNaimtVn+KVKOKxiyq9qZWY9UMscm++31/f1wJcW/2TXJv8nw/n3ySvOfc856b4155ct73HCE6OloCEREREREREaVLltcdICIiIiIiInIWDNFEREREREREGcQQTURERERERJRBDNFEREREREREGcQQTURERERERJRBDNFEREREREREGcQQTURERERERJRBDNFEREREREREGcQQTURERERERJRBDNFEREREREREGcQQTURERERERJRBDNHkMHQ6Ha5duwadTpfXXaFnOCaOhePheDgmjodj4ng4Jo6F4+F4OCbOhyGaHIrZbM7rLtBLOCaOhePheDgmjodj4ng4Jo6F4+F4OCbOhSGaiIiIiIiIKIMYoomIiIiIiIgyiCGaiIiIiIiIKIMYoomIiIiIiIgyiCGaiIiIiIiIKIMYoomIiIiIiIgySJHXHSAiIiIioudEUURCQkKO7BssiiJUKhViYmIQFxdn9/Yp8zgmOUuj0cDNzQ0ymf3mjxmiiYiIiIgchCiKePz4Mdzd3VG0aFEIgmD39g0GA1QqlV1DBWUdxyTnSJIEnU6Hx48fw9vb224/X44SOSSTKa97QERERJT7EhIS4O7uDhcXF7sHaKKCRhAEuLi4wN3dHQkJCXZrlyGaHEZMjIDffy+G1q2LYPJkTV53h4iIiCjX6XQ6aDT8PYjInjQajV1vj+Dl3JTnDhyQY+VKFcLDPaHTWf7ieuuWiGnTdFCp8rhzRERERLmMM9BE9mXv1xRnoinPhYaqsHatKjlAA8DjxzJs386/8RARERERkWNhiKY8FxJisHk8NJTT0ERERERE5FgYoinP1a9vRkCA2er4rl0K3LvHy5mIiIiIKOd4eXmhbdu22Wpj//798PLywuzZs+3UK3JkDNGU5wQBCAkxWh03mwWEhSnzoEdERERElJu8vLwy9UHpq1atGnx9ffO6G/kSbzolh9CzpwGzZqkhSSlnnkNDVfjoIwO4vgYRERFR/jVhwgSrY0uXLkVsbKzNMns6duwYXFxcstXG66+/jmPHjsHb29tOvSJHxhBNDqFUKQmNGhmwd686xfHLl+U4cUKON9+0vtybiIiIiPKHTz75xOrYL7/8gtjYWJtl9lSxYsVst+Hq6mqXdsg58HJuchg9e2ptHl+9mguMERERERFw8+ZNeHl5YejQobh06RJCQkJQtmxZeHl54ebNmwCATZs2YcCAAahZsyZKlCgBf39/tG7dGn/88YfNNm3dEz106FB4eXnhxo0b+O677/Dmm2/Cx8cHVatWxZw5cyCKYor6qd0TXa1aNVSrVg3x8fGYMGECAgMD4ePjg3r16qXan5s3b6J///4ICAiAn58f2rRpg4MHD2L27Nnw8vLC/v37s/rjS1VCQgJmzZqFN998E76+vggICEBwcDCOHDliVVen02HRokWoX78+/P39UbJkSVSrVg3vvvsuzp07l1xPFEWsXLkSTZs2RUBAAIoXL47KlSuje/fuOfIcchNnoslhvPOOHh4eJsTFpfxnuX69ErNna+HqmkcdIyIiIiKHcv36dbRo0QKVK1dGr1698OTJE6hUlomX6dOnQ6lUok6dOihevDgePXqEbdu2oV+/fvjiiy8wePDgDJ/ns88+w8GDB9GqVSs0bdoUW7ZswZw5c2A0GjF58uQMtWEymdC5c2dER0cjKCgIWq0W69evx7vvvovff/8dTZo0Sa4bFRWFVq1a4d69e2jevDmqV6+OK1euoFOnTmjUqFGmfkYZpdPp0L59e5w8eRI1atTA0KFD8eDBA2zYsAF//fUXli9fjo4dOybXHzp0KDZs2IAqVaqgV69eUKvVuHPnDvbv349//vkH1apVAwBMmzYNX3/9NcqWLYtu3brB3d0dUVFROHLkCPbu3YuGDRvmyPPJDQzR5DA0GqBlyyf4/XefFMfj4gRs2qRE9+7Wi48RERERFSQtWrhl6/GSBEiSCEGQ2X3NmZ07E+zbYBqOHDmC8ePHY+LEiVZlv/32GwICAlIci4+PR8uWLfH555+jT58+cM3g7MyZM2dw8OBBFC9eHAAwfvx41KpVC8uWLcOECROSg3ta7t69i5o1a2Lz5s3J9bt164YOHTrg22+/TRGip06dinv37mHy5MkYM2ZM8vFVq1Zh+PDhGepzZn399dc4efIkgoOD8f3330N49g9j8ODBaNGiBUaMGIFmzZrBw8MDMTEx2LhxI1577TX89ddfkMvlye2YzWbExcUlf79y5UqUKFECBw8etPp5P336NEeeS25hiCaH0r79I6sQDVgu6WaIJiIiooLu+HH++g4Avr6+GDt2rM2ylwM0ALi7u6NXr16YNGkSTp06hQYNGmToPOPGjUsO0ADg7e2NNm3aYM2aNbhy5QqqVKmSoXZmzZqVInA3btwYpUuXxqlTp5KP6fV6/PHHHyhWrBiGDRuW4vG9e/fGN998gytXrmTofJmxZs0aKJVKTJkyJTlAA0CNGjXQs2dP/Pzzz9iyZQt69OgBQRAgSRI0Gg1kspR3BsvlcquV05VKZYqgnaRw4cJ2fx65ifdEk0N59dVEBAZah+X9+xW4cYNLdBMRERERULVq1VRngR8+fIiJEyeidu3aKFGiRPK2WJMmTQIA3Lt3L8Pnee2116yO+fn5AQBiYmIy1EahQoVsBns/P78Ubfz333/Q6/WoWbMm1OqUi+0KgoDatWtnuN8ZFRsbixs3bqBcuXLJz+tFSZdcJ93r7OnpiZYtW+LIkSNo1KgR5s+fj6NHj8JotP79vUuXLrh16xbq1q2LmTNnYu/evdBqba+B5GwYosmhCALQo4fOZtmaNVxgjIiIiIiAYsWK2Tz+9OlTvP3221iyZAmKFCmC3r17Y+zYsZgwYQLatGkDwDLjm1EeHh5Wx5JmVs3mjO0e4+npafO4XC5PsUBZ0qXQRYsWtVnfx8f6as3sSjpnaj/PpH2mX7xMe8WKFRg7dixiY2MxY8YMtGrVCuXLl8fHH3+MxMTE5Hpz5szBjBkzoFKpMG/ePHTo0AFly5bFkCFD8PjxY7s/l9zEEE0Op0sXLRQKyer4L7+o8NJCiERERERUAAmp3NC9atUq3L59G59++im2b9+OuXPnYtKkSfjkk0/w5ptv5nIvMycpsD969Mhm+YMHD3LsnA8fPkzznC/+McHV1RWTJk3CmTNncPr0aSxatAgVKlTAd999l+IedYVCgeHDh+PIkSO4cOECfvzxR9StWxe//vorBg4caPfnkpt4UwU5nGLFJLRqZcKWLcoUxyMjZdi/X47GjblnNBERERVMb75pytbjc3JhMUdw/fp1AEiedX7R4cOHc7s7mVKhQgWo1WqcPn0aer0+xSXdkiTh+PHjdj+np6cnAgICcO3aNURFRaFkyZIpyg8cOAAAyStuvywgIAABAQHo2rUrXnnlFWzbtg0LFy60qleiRAl07doVnTt3xhtvvIE9e/ZAq9XCxcXF7s8pNzBEk0MKCTFYhWgACA1VoXHj/HEvBREREVFmZXcFbFEUYTAYoFKprBaGyg9Kly4NwLJ694uLfv3222/YsWNHXnUrQ9RqNTp06IC1a9di6dKlGDlyZHLZmjVrcPny5Rw5b8+ePTF79mxMmzYN3333XfIs//nz5/HLL7/A09MzeR/tR48e4cGDB6hcuXKKNqKjo6HX61GkSBEAlkvmT58+jbfeeitFvYSEBCQkJECpVDr1vz+GaHJILVqY4OMj4sGDlC+u8HAlvvxSi5cW/iMiIiIiQvfu3bFw4UKMHz8e+/fvR+nSpXH+/Hns3bsXQUFB2LRpU153MU2fffYZ9uzZg6lTp+LgwYPJ+0T/+eefaN68OXbt2pWp8Gk0GjF06NBUy5cuXYoRI0Zgx44dCAsLw+XLl9G4cWM8fPgQGzZsgMlkwvfff598OXdUVBQaNWqEqlWrokqVKihZsiSePHmCrVu3wmg0Jm/DpdVq0apVK1SoUAGvvfYaSpUqhYSEBGzfvh3379/H8OHDrRZPcyYM0eSQlEqge3cjFi1K+eLS6QRs2KBC//6GPOoZERERETkqPz8/bNmyBVOmTMGePXtgNptRvXp1bNiwAbdv33b4EF2qVCns2LEDU6dOxe7du3Hw4EHUqFED69evx8aNGwHYXuwsNaIoYs2aNamWL126FBqNBuHh4Vi4cCE2bNiAJUuWwMXFBfXr18fo0aNRt27d5Pr+/v74+OOPsW/fPuzduxdPnjyBt7c3atSogSFDhqB58+YAADc3N0ybNg179+7F4cOH8fDhQ3h5eaFChQqYMmUKunTpkrUfkIMQoqOjrVdwIsoDOp0OkZGRKF26NDQaDS5elKFOHes3iTfeMGHXruxdykQZ8/KYUN7ieDgejonj4Zg4Ho5J5jx8+DDVlZLtIb9fzu2MMjom77zzDo4dO4Zbt27B3d09F3uYP9jztcVXDjmswEARb7xhvXjGiRMKXLzIf7pERERElP/Y2sc6LCwMR44cQZMmTRigHQAv5yaHFhJixIkT1v9MQ0NVmDHD9n7SRERERETOqm7duqhevToqVaoEuVyOc+fO4cCBA/Dw8MCMGTPyunsEzkSTg+vc2QCNxvqOg7AwJYzGPOgQEREREVEOeu+99/Dw4UP8+uuv+OGHH/Dff/+hW7du+Ouvv1KsOE55hzPR5NAKFQLatzdi7VpViuMPHsiwc6cCbdpkb69EIiIiIiJHMnnyZEyePDmvu0Fp4Ew0ObyQENsrcYeGqmweJyIiIiIiyikM0eTwGjY0o3Rp0er4n38q8PChkAc9IiIiIiKigoohmhyeTAb06mU9G20yCQgLU+ZBj4iIiIiIqKBiiCan0LOn7Uu6f/lFBYk7nRMRERERUS5hiCanEBAgoWFD60XEIiLkOH1angc9IiIiIiKigoghmpxG7962Z6NXr+Yl3URERERElDsYoslpBAUZ4elpfe32unUq6HR50CEiIiIiIipwGKLJabi6Ap06Ga2Ox8QI2LKFs9FERERERJTzGKLJqfCSbiIiIiIiyksM0eRU3njDjIoVzVbH9+xRIDKSe0YTERERkW2hoaHw8vJCaGhoiuPVqlVDtWrVst2OPc2ePRteXl7Yv39/jp2Dso4hmpyKIAAhIdaz0ZIk4NdfVXnQIyIiIiLKrvfffx9eXl5Yt25dmvViY2NRokQJ+Pv7Q6vV5lLv7G///v3w8vLC7Nmz87orGZIU6n///fe87opDYIgmp9O9uxFyufUCY6GhSohiHnSIiIiIiLKlT58+AIDVq1enWe/333+HVqtF165d4eLiYpdzh4eHIzw83C5t2cugQYNw7NgxvP7663ndFbKBIZqcTvHiEpo3t94z+sYNOQ4d4p7RRERERM6mUaNGKFOmDPbt24fIyMhU6yWF7KTQbQ9ly5ZF2bJl7daePXh7e6NixYpwdXXN666QDQzR5JRsXdINAKtX85JuIiIiImcjCAJCQkIgimKq9xpfuHABJ0+eRJUqVVCzZk3ExMRg4cKFaNOmDQIDA1GsWDEEBgZi8ODBuH79eobPndo90U+fPsWoUaPwyiuvoESJEnj77bexadOmVNtZtWoVevbsiWrVqsHX1xcBAQHo3Lkz9u3bl6Le7NmzERQUBAD44osvUKRIERQvXhxFihTBzZs3k+ukdk/0tm3b0K5dO/j7+6N48eKoX78+Fi9eDJMp5STTzZs34eXlhaFDh+LatWsICQlBmTJlULJkSXTo0AHnzp3L8M8oszLaRwDYt28funbtisDAQPj4+OCVV15B69atsWLFihT1Tp8+jb59+6Jq1arw8fFB+fLl8fbbb2PevHk59jxSo8j1MxLZwTvvmODtLeLx45R/BwoPV2LuXC08PPKoY0REREQ5yK1Fi+w1IEkQJQkyQbAsNmNHCTt3ZuvxvXr1wpw5c/DLL79gwoQJEF7qX1K4TpqFvnz5MmbNmoWGDRuiXbt2cHV1xeXLl7Fu3Trs2LEDe/fuhb+/f5b6kpiYiLZt2yIiIgK1a9dG/fr1cefOHbz33nto2rSpzceMGzcOVatWRZMmTVC0aFFERUVh69at6NixI1atWoW2bdsCABo0aIBbt25hzZo1qF+/PurXrw+z2Qy5XI5ChQql2a/Fixdj0qRJKFy4MLp27QpXV1ds27YNkyZNwuHDh7F69Wqrn9utW7fQvHlzBAYGonfv3rh+/Tq2bt2KoKAgHDt2DD4+Pln6Gdmjj3/++Sd69OiBQoUKoU2bNihevDgePXqE8+fPIywsDO+++y4A4OzZs2jVqhXkcjnatGmD0qVLIyYmBhcvXsSKFSswduxYuz6H9DBEk1NSqYDgYCOWLlWnOJ6YKGDDBiX69rXeT5qIiIjI2SmOH8/rLuSYUqVKoWnTpti1axf27duHxo0bJ5eZTCasXbsWarUa3bt3BwBUrFgRly5dQuHChVO0s2/fPnTs2BHz5s3DN998k6W+fP3114iIiEC/fv3w9ddfJx/v3r07unTpYvMxR44cQUBAQIpj9+7dw9tvv43PPvssOUQ3bNgQALBmzRo0aNAAEyZMgMFggEqlgkyW+oXC169fx9SpU1GsWDH8/fffKFWqFABg8uTJ6NixI7Zs2YKwsDD06NEjxeMOHjyIqVOnYuTIkcnHZs6ciXnz5iE0NBSjRo3K8M8lPZnt4+rVqyFJEjZt2mR1NcCTJ0+Svw4LC4Ner0doaGjyz9FWvdzCy7nJaaV2SXdoKC/pJiIiInJGqS0wtn37djx48ABt2rRJDs2FChWyCtCA5f7qwMBA7NmzJ8v9+PXXX6FSqTBx4sQUx5s1a5Yi3L/o5QANAMWLF0dQUBCuXr2KW7duZbk/APDbb7/BZDJh2LBhyeEUANRqNaZOnQoA+OWXX6weV6ZMGXz00UcpjiX9nE+dOpWtPtmrj7YWiStSpEiW6+U0hmhyWlWriqhRw3rP6KNHFbhyhf+0iYiIiJxNmzZtULRoUWzevBkxMTHJx1NbUGz//v3o1asXKlWqhKJFi8LLywteXl6IiIjAvXv3stSH2NhY3Lx5E+XKlYOvr69Ved26dW0+7saNG/joo4/w2muvwdfXN7kvy5YtA4As9yfJ2bNnAVguB39Z7dq1odFobN7nXK1aNasZbj8/PwBI8TO2h8z2MWlWv3nz5hg3bhw2bdqEx48fWz22U6dOkMlk6N27Nz788EOsW7cOUVFRdu17ZjBpkFPr3dv2bPQvvyhzuSdERERElF1KpRLdu3eHVqtN3jP6/v372LVrF0qVKoUmTZok1924cSPat2+P/fv3o06dOhg6dCjGjx+PCRMmoHTp0jAYbP+emJ64uDgAQNGiRW2W27qH+Nq1a2jSpAlCQ0MREBCA/v37Y9y4cZgwYQLq168PANDr9Vnqz8v9KlasmFWZIAgoVqxYcp0XedhYLEihsNzVazZbT0jlZh87duyI0NBQVK5cGT/99BP69OmDChUqoH379smBHADeeOMNbN68GfXq1cO6devw/vvvo3LlymjatKnVwm25gfdEk1Pr2tWITz/VwGBIuYDCmjUqfPqpHgr+CyciIqJ8xPTmm9lrIAcXFrOXPn364Ntvv8WqVaswYMAAhIWFwWQyISQkJMWM6pw5c6DRaLBnzx6UL18+RRvr16/P8vmTQuejR49slj948MDq2JIlSxAdHY3vv/8++Z7tJKNGjcLBgwez3J+X+/Xw4UOrBdMkScLDhw9tBubclJU+tm3bFm3btkVcXByOHj2KTZs2YdWqVejatSuOHTsGLy8vAEC9evVQr149aLVanDhxAtu3b8fy5cvRvXt3HD582Obl9DmFEYOcWuHCEtq2NWLDhpT3Qd+7J8Pu3Qq0bGm9jD4RERGRs8ruCtiiKGZoEau8FBgYiDfffBPHjx/H+fPnERoamrwF1ouuX7+OwMBAqwB979493LhxI8vn9/T0RJkyZXDt2jXcv3/f6pLuw4cPWz0maUutNm3apDguSRKOHj1qVV8ulwPI3Exw9erVsXnzZhw4cACvv/56irITJ05Ap9Ohdu3aGW4vJ2Snjx4eHmjevDmaN28Os9mM1atX4+TJk2jWrFmKei4uLmjYsCEaNmyIQoUKYdasWfj777/Rv3//HHteL3PMVw5RJvTubXslbi4wRkREROScku59Hjt2LC5duoQmTZpYzWyWLl0a169fTzEzrNPpMHr0aBiN2duppXv37jAYDJg1a1aK47t378bevXut6pcuXRqAZYXuF3311VeIiIiwqp+0INqdO3cy3Kdu3bpBoVDg22+/xd27d5OPGwwGTJkyBYBlm7C8lNk+Hjx40OYfEh4+fAjAsiAZABw7dgw6nS7dermFM9Hk9Jo0McHPT8SdOyn/JrR1qwKPHwvw9pbyqGdERERElBWdOnXCJ598khxKX15QDAAGDRqE8ePHo1GjRmjfvj3MZjP+/vtvSJKEqlWr4vz581k+/4gRI7B582b8/PPPuHjxIurVq4c7d+5gw4YNaNWqFf78888U9fv374/Q0FD07dsXHTt2RJEiRXDixAmcOXPGZv2KFSuiRIkSWL9+PVQqFXx9faFQKDB48OBU94ouW7Yspk6dikmTJqF+/fro1KkTXF1dsX37dly5cgVt2rSxupTc3pYvX45du3bZLOvbty/q1q2bqT5OmDAB9+7dQ506deDv7w9BEHDkyBGcPHkSb775ZvIibgsXLsSBAwdQt25dlClTBhqNBmfOnMHevXsREBCAdu3a5ejzfhlDNDk9uRzo0cOA+fM1KY4bjQJ++02JIUOytqgEEREREeUNDw+P5EWnChcubLU3MAAMHDgQSqUSy5Ytw8qVK1GoUCG0bNkSU6ZMQb9+/bJ1fjc3N2zZsgXTpk3D5s2bcebMGQQGBuKnn35CbGysVSiuUaMG1q9fj88//xybN2+GTCbDW2+9he3bt2Pbtm1W9eVyOVatWoUpU6bg999/R3x8PADLDHhqIRoAhg0bhnLlyuHbb7/F2rVrYTAYUL58ecycORNDhgyBkMP3uR86dAiHDh2yWdagQQPUrVs3U30cPXo0Nm3ahNOnT2P37t1QKBTw9/fHtGnTMGDAgOTL3gcMGABPT0+cPHkShw4dgiRJKFWqFMaMGYMPPvgAnp6eOfq8XyZER0dzmo4cgk6nQ2RkJEqXLg2NRpP+A15w7ZoMtWpZL6RQrZoZ+/fH26uLBU52xoTsj+PheDgmjodj4ng4Jpnz8OFDmysb24sz3BNd0HBMcoc9X1scJcoXypUTUbeu9SJi587JceYM/5kTEREREZF9MF1QvhESYvuybS4wRkRERERE9sIQTflGx45GuLlZ353w229KZHNveyIiIiIiIgAM0ZSPuLtbgvTLnj6VYds2rqFHRERERETZxxBN+Qov6SYiIiIiopzEEE35St26ZpQrZ71h+19/KRAVlbNL/hMRERERUf7HEE35iiAAISHWl3SLooBff+VsNBERERERZQ9DNOU7PXoYIJNZLzAWGqqExF3RiYiIiIgoGxiiKd/x85PQtKn1ntFXr8px9Kg8D3pERERElHES/+pPZFf2fk0xRFO+ZOuSbgBYvZqXdBMREZHj0mg00Ol0ed0NonxFp9NBo9HYrT2GaMqXWrc2wstLtDq+caMSCQl50CEiIiKiDHBzc0N8fDy0Wi1npImySZIkaLVaxMfHw83NzW7tcvNcypc0GqBbNyN++EGd4nh8vIA//lCiVy/bM9VEREREeUkmk8Hb2xsJCQl49OiR3dsXRTF5Vk4m43yaI+CY5CyNRgNvb2+7/mwZoinfCgkxWIVowHJJN0M0EREROSqZTAYPDw94eHjYvW2dTofY2Fj4+vra9fJWyjqOifPhnzrygfDwcHTs2BEBAQHw8vLCzZs387pLDqFGDRFVqljvGX3okALXr/OfPhERERERZR6TRD6QmJiIevXqYeLEiXndFYciCEDv3gabZaGhylzuDRERERER5QcM0flAjx49MH78eDRo0CCvu+JwgoONUCqtF+VYs0YFs/UkNRERERERUZqcNkQvXLgQXl5e8PLywvHjx3P9/GFhYRg5ciSaNGkCHx8feHl5ITQ0NM3HnDp1Ct26dYO/vz9KliyJ5s2bY8OGDbnU44LJ21vCO+9Y7xl9544Me/dySQAiIiIiIsocpwzRERERmD17tl2XKc+smTNnYsWKFYiMjISvr2+69fft24dWrVrhyJEj6NSpE/r374/79++jf//+WLRoUS70uODiJd1ERERERGQvTheijUYjhg4dimrVqqFt27aZeuyvv/6KW7dupVpuNpuxePFiGAy2Q9eLFi1ahLNnz+Lq1at477330qxrMpkwYsQIyGQybNmyBV9//TU+//xzHDhwABUqVMCMGTOs+jV16tTkmfbUPihjmjUzoXhx6z2jN29WIjo69/tDRERERETOy+lC9Lx583Dx4kUsXrwYcrk8w4+7c+cORowYgXbt2tkM0qIoYujQoZg0aRJWrFiRbntNmjSBv79/hs69b98+XL9+HV27dkX16tWTjxcqVAijR4+GwWDAmjVrUjxm2LBhOHbsWJoflDEKBdC9u/WWVnq9gHXrVHnQIyIiIiIiclZOFaJPnz6N+fPnY8KECQgMDMzUY/38/PDjjz8iKioKQUFBiIyMTC4TRRFDhgzB2rVr0aNHD7z//vt27feBAwcAAE2bNrUqa9asGQDg4MGDKY4XLVoUFStWTPODMi4khJd0ExERERFR9jlNiNbr9cmXcY8YMSJLbQQFBWH58uW4c+cO2rVrh8jIyOQZ6LVr1yI4OBhLliyBTGbfH8vVq1cBAOXLl7cq8/X1hbu7O65du5bl9p8+fYqzZ8/iypUrAIBLly7h7NmzePr0aZbbzG8qVhRRu7b1AmP//KPAv/86zcuAiIiIiIjymNOkh1mzZuHq1av49ttvM3UZ98s6dOiAZcuW4fbt2wgKCsKAAQMQFhaGLl26YOnSpXYP0AAQGxsLAPD09LRZ7uHhkVwnK7Zu3YpGjRqhX79+AIDg4GA0atQIW7duzXKb+VHqs9G8pJuIiIiIiDLGKUL0sWPHsGjRIowdOxaVK1fOdnudO3fG0qVLcePGDWzYsAFt27bFsmXLshXO81JISAiio6OtPkJCQvK6aw6lUycjXFys94xeu1aJDKwlR0RERERE5Pgh2mQyYejQoahSpQpGjRpllzYlScK+ffuSv7948SIePHhgl7ZtSZqBTm22OS4uLtVZarIfT0+gfXvrBcYePZLhzz+5ZzQREREREaXP4UN0fHw8rl69inPnzqFYsWIptnhKWtG6RYsW8PLywubNm9NtT5IkjBgxAqtXr0bnzp2xbNky3LhxA0FBQbh7926OPIeke6GT7o1+0f379xEfH49y5crlyLkpJV7STURERERE2eHw029qtRp9+vSxWXbo0CFcvXoVrVu3RtGiRdPdckqSJIwcORIrV65Ep06d8MMPP0Aul0Mmk2Hw4MEICgrC5s2bUbx4cbs+h/r162PBggXYvXs3unTpkqLsr7/+Sq5DOa9BAzPKlBFx82bKvx/t3KnA/fsCfH2tL/cmIiIiIiJK4vAh2sXFBYsWLbJZNnToUFy9ehWjR4/Gm2++mWY7kiRh9OjR+Pnnn1MEaADo2rUrAKQI0r6+vnZ7Do0bN0ZAQADWrVuHwYMHJ+8VHRMTgwULFkClUqFHjx52O59TkyS4/O9/EBo1glilit2bl8mAXr0MmD1bk+K42SwgLEyJjz7izdFERERERJQ6hw/R9nL37l1s3rwZHTt2xA8//ACFIuVT79q1KyRJwpAhQ7Bnzx507949zfZWrlyJw4cPAwAiIiIAAKtWrUreE7pu3bro27cvAEChUOCbb75Bly5d0LZtW3Tu3Bnu7u4IDw9HZGQkZsyYgTJlytj7KTsdIT4eZSdNgueOHTBXqID43bstNzLbWc+eBsyZo4YkCSmOh4aqMHy4AYKQygOJiIiIiKjAKzAhumTJkti5cydKlSplFaCTdOvWDTVr1kSFChXSbe/w4cPJ92QnOXLkCI4cOZL8fVKIBoBGjRph+/btmD17NjZs2ACj0YjKlStj2rRp6Ny5cxafVf4hi4hAkb59ofjvPwCA/L//4DJiBLQ//QR7p1p/fwmNG5uwZ48yxfFLl+Q4eVKON94w2/V8RERERESUfwjR0dG8CZTylPzECbgFBUHQaq3KtF9+CcOgQXY/52+/KTFwoKvV8Xff1WPhQp3dz+esdDodIiMjUbp0aWg0mvQfQDmK4+F4OCaOh2PieDgmjoXj4Xg4Js7H4VfnpvzPXL06zFWr2izTfPop5CdP2v2c7doZ4elp/fej9etVSEy0++mIiIiIiCifYIimvKdSIfGnnyAWLmxVJBiNcH33XQhPn9r1lC4uQNeu1ouIxcYK2LxZaeMRREREREREDNHkIKTSpRGzeLHNMllkJFyGDAFE0a7nDAkx2jy+ejX3jCYiIiIiItsYoslhGJo1Q9R779ksU/75J9Rff23X89WqZUZgoPUiYvv2KXDzJpfoJiIiIiIiawzR5FCiBg2CoX59m2XqGTMgf7aFmD0IAhASYntf6DVrOBtNRERERETWGKLJscjliFm6FKKvr1WRIIpwHTAAwv37djtd9+5GyOXWC4yFhqrsffU4ERERERHlAwzR5HBEHx8kLl8OSWb9z1N2/z5c338fMJnsci4fHwktW1q3FRkpw/79crucg4iIiIiI8g+GaHJI5gYNoJ882WaZYv9+qGfPttu5eve2fUl3aCgv6SYiIiIiopQYoslh6UeMgLFVK5tlmvnzodixwy7nadnShGLFrK/dDg9XIibGLqfIcbIzZ+DaqRM8atSAy7BhQHR0XneJiIiIiChfYogmxyWTQfvddxBLl7ZZ7DJ4MITIyGyfRqkEgoOtt7vS6QRs2OD4e0bL9+6F+zvvQPn335DdvAnV6tVw69kTMFuvPE5ERERERNnDEE0OTSpcGIkrVkBSWodZ2dOncO3fHzDYvhw7M1JbpdvRL+mW79kDt+7dIWi1KY4rDh+G6qef8qZTRERERET5GEM0OTzz669D9/nnNssUJ05Ak8q905lRubKIWrWsFxg7flyBS5cc82Wi2L0bbj16QNDpbJZrZsyAcPduLveKiIiIiCh/c8x0QPQSw8CBMHTqZLNM/f33UGzcmO1zhIRYX9INOOZstOKvv+Das2eqARoAhNhYaCZOzMVeERERERHlfwzR5BwEAdpvvoG5QgWbxa7Dh0P233/ZOkWXLgZoNNZ7Rv/6qxJG2/k6Tyh27oRrr14Q9Pp066o2bIBi585c6BURERERUcHAEE3Ow8MDiT//DMnFxapIiIuDa79+wEv3BmeGlxfQrp11Wn7wQIZduxRZbteeFDt2wDUkJEMBOonLmDFAYmIO9oqIiIiIqOBgiCanIlapAu28eTbL5P/+C5dx47LVviNf0q3Yvh2uvXtDSGUhNUO3bhC9vKyOy27dgnru3BzuHRERERFRwcAQTU7HGBICQ+/eNstUq1dDGRqa5bYbNTKhVCnrPaO3b1fg0SMhy+1ml2LbNrj26ZN6gO7bF9rvv4du2jSb5epFiyCLiMjJLhIRERERFQgM0eSUtHPnwlylis0yl7FjIfv33yy1K5cDPXtaB1WTSUBYWN7sGa3YsgWufftCSOXGbEO/ftAuXAjIZDD26QNTnTpWdQSTCS6jRgGi9R8IiIiIiIgo4xiiyTm5uCBx5UpIHh5WRYJWa7k/OjY2S02ntWe0ZL3uWI5SbNoE1379Ug3Q+v79of3qK0D27KUsk0G7YAEkhfU93IqjR6FctSonu0tERERElO8xRJPTEsuXR+LixTbL5P/9B5cRI5CV1BsQIKFBA+s9oyMi5DhzJvdeMorwcLj27w/BZN0XANAPGADd/PnPA/QzYuXK0A8fbvMxLp99BuHBA7v3lYiIiIiooGCIJqdm6tAB+iFDbJapNmyAatmyLLWb2mz06tW5s8CY4o8/0g7QAwdCN2+eVYBOLh83DmKZMlbHhZgYaCZNsmtfiYiIiIgKEoZocnq66dNhevNNm2WaSZMgP3Ei0222b2+Eh4f1LPZvv6mg02W6uUxRbtgA1/feg2A22yzXDxoE3ZdfAkIaC525uqa6irlq7VrI9+zJfkeJiIiIiAoghmhyfioVEn/6CWLhwlZFgtEI13ffhfDkSaaadHMDOnWyvg85JkbA1q05t8CY8vff4fL++6kH6CFDoPvii7QD9DOmFi1g6NTJZpnL6NHI8b8GEBERERHlQwzRlC9IpUtDm8ql27Lbt+EyZEimV6ZO/ZLunAnRyt9+g8vAgakH6KFDoZs9O0MBOolu9mxInp5Wx+XXrkE9f36W+0pEREREVFAxRFO+YWrRArqxY22WKXfsgHrhwky1V7u2Ga+8Yh1o//5bgVu37LtntHLtWrgMHgwhlaCv//BD6GbNylSABgCpeHHoJk+2WaZeuBCyy5cz3VciIiIiooKMIZryFf0nn8DUsKHNMvXMmZDv35/htgTB9my0JAlo0sQdCxaoEReX5a4mU/76K1yGDEk9QA8fDt3MmZkO0EkM770H0+uvWx0XjEbL3tG5vW8XEREREZETY4im/EUuR+KPP0L09bUqEkQRrgMGQLh/P8PNde9uhExmHTKfPJFh+nQNqlf3wLx56qxuSQ3lL7/AZejQVAO0buRI6KZPz3KABgDI5dB+9RUkudyqSHHwIJS//JL1tomIiIiIChiGaMp3JF9fJC5fDsnG9k+yBw/gOmAAkMrWUS8rUUJCixap1336VIaZMy1h+ssv1YiJyXg/laGhcPnwQwipzATrRo+GfsqU7AXoZ8Tq1WFIZSswzeTJEB4/zvY5iIiIiIgKAoZoypfMDRpAn8q9wIoDB6CePTvDbX3yiQ4qVdqXPEdHyzBrlgbVq3tizhw1oqPTblO5ahVchg1LPUCPHWvpvx0CdHKbn3wCsVQpq+OyJ0+g+ewzu52HiIiIiCg/Y4imfEs/YgSMrVrZLNPMnw/Fjh0Zaue110Ts2BGPevXSn72OiREwZ44lTM+ebTtMK1euhOvw4akH6HHjoP/0U7sGaACAuzu0X35ps0gVGgr5gQP2PR8RERERUT7EEE35l0wG7XffQSxd2maxy+DBECIjM9TUa6+J2Lo1AZs2xaNBg/TDdGysgC++sITpzz9X4+lTSyBWrlgB148+SvVxugkTciZAP2Nq0wbGtm1tlrmMHg3o9TlyXiIiIiKi/IIhmvI1qXBhJK5YAUlpvbez7OlTuPbvDxhs7wdtS8OGZmzenIAtW+LRqFHGwvTcuZZ7pnd1WwXXkSNTrav7+GPoP/kkw33JKu0XX0Byd7c6Lr98Gepvvsnx8xMREREROTOGaMr3zK+/Dt3nn9ssU5w4AU0q906npX59M8LDE7BtWzyaNDGmWz8k7jt02Tk81XLdxInQf/xxpvuRFVKpUtBNnGizTD1vHmTXruVKP4iIiIiInBFDNBUIhoEDYejUyWaZ+vvvodi4MUvt1q1rxsaNifjzz3g0bWo7TH+Ab7EUH6Tahm7SJOjHj8/S+bPKMGgQzNWrWx0X9HpoRo/m3tFERERERKlgiKaCQRCg/eYbmCtUsFnsOnw4ZP/9l+Xm33rLjPXrE7FjRzyaN38epodhEb7FsFQfN0X5OSbETsLDhzlzD3SqFApoFy6EZOPea+WePVCuW5e7/SEiIiIichIM0VRweHgg8eefIbm4WBUJcXFw7dsXSEzM1ilq1zZj3bpE7NoVjyWVFmARUl9EbALmYLpxIr75Ro0aNTwwaZIGDx7kXpg216oFw8CBNss0Eyci3X26iIiIiIgKIIZoKlDEKlWgnTfPZpk8IgIu48bZ5Tz1jn6DoZfGpFo+Dl/iS0xI/j4xUcDixZYwPXGiBvfv506Y1k2aBLFECavjsocPoZk6NVf6QERERETkTBiiqcAxhoTA0Lu3zTJVaCiUq1dnq33VokVw+fTTVMvHYB7mwXZY12oFLFliCdMff6zBvXs5HKY9PaGdM8dmkXrFCiiPH8/Z8xMREREROZkcD9HR0dGIiIiAnvvPkgPRzp0Lc5UqNstcxo6F7Pz5LLWr+uYbuKSx2vfVD2fhUtvUL/FOotMJ+O47S5geP16DqKicC9Om9u1hbNnSZpnHuHEQTOlv5UVEREREVFBkO0SfOXMGn3/+OXbv3p3iuFarxYABA1CuXDk0aNAAgYGB+OOPP7J7OiL7cHFB4sqVkDw8rIoEnQ6u/foBsbGZalK1cCFcPvss1XLt7Nko+vkHCA1NxL59cQgKSn9rLL1ewLJlatSs6YFx4zS4cycHwrQgQDt3rs17xZUXL8I3NNT+5yQiIiIiclLZDtGrV6/G/PnzIb20Jc6sWbOwfv16SJIESZIQHR2NgQMHIiIiIrunJLILsXx5JC5ebLNMfvUqXD76KMNbPakXLIBLGvcQa7/4AoahQ5O/r15dxKpViThwIA4dOmQsTP/wgyVMjxmjwe3b9g3TUpky0KWyT3WJH36A7OZNu56PiIiIiMhZZTtEHzp0CBqNBm+//XbyMYPBgJ9//hlKpRJr167FjRs3MHjwYBiNRnz33XfZPSWR3Zg6dIB+8GCbZaqNG6FatizdNtTz5kEzfXqq5dq5c2FI5RxVq4r4+edEHDoUh06dDBCEtEO7wSBg+XJLmB41SoNbt+wXpg0ffABz5cpWx+V6PTwnTuTe0UREREREsEOIfvDgAUqUKAGZ7HlTx44dQ1xcHFq3bo0WLVqgUKFCmDJlCtzc3HDw4MHsnpLIrnQzZsD0xhs2yzSTJkF+4kSqj1V/+SU0M2emWq6dNy/VbaReVLmyiP/9T4tDh+LRpUv6YdpoFPC//6nx+useGDHCBTdv2iFMK5XQLlxos0j9119QhIdn/xxERERERE4u2yE6OjoahQsXTnHs2LFjEAQBzZo1Sz7m4uKCgIAAREVFZfeURPalUiHxf/+D+NK/YwAQjEa4vvsuhCdPrMrUc+ZAM2tWqs1qFyyA4f33M9WVV18VsXy5FkeOxKNbNwNksvTD9M8/q/D66x4YPtwFN25kL0yba9eGvn9/m2UuEyYAMTHZap+IiIiIyNllO0S7uLjg0aNHKY4dPnwYAPDWW2+lOK5SqVLMWBM5Cql0aWhTuXRbdvs2XIYMAUTxWWUJ6lmzoEllaygASFy4EIb33styfypVEvHDD1ocPRqP4OD0w7TJJGDVKkuYHjbMJVuXeeumTIFYrJjVcdm9e2nOuhMRERERFQTZTrQVK1bErVu3cOHCBQDA48ePsX//fnh7e6NSpUop6t69exdFixbN7imJcoSpRQvoxoyxWabcsQPqhQufB+gvv7RZTxIEJH7zDYzvvmuXPr3yiohly7Q4diwePXqkH6bNZgGrV6vwxhuWfaYfPsxCmPbygm72bJtFqh9/hPzUqcy3SURERESUT2Q7RHfs2BGSJKFbt2749NNPERQUBIPBgM6dO6eoFxkZiXv37qFcuXLZPSVRjtF/8glMDRrYLFPPnAmX99+HZu5cm+WSIED7zTcw9u1r935VqCDiu++0OHEiHr16GSCXp78A2XffWRYgmzVLndndumDs0gXGFxYLTCJIElxGjAC4dzQRERERFVDZDtGDBg1CvXr1cOfOHSxZsgQXLlxAhQoVMGHChBT1NmzYAABo2LBhdk9JlHMUCiQuXw7R19eqSBBFqH7/3ebDJEGAdvFiGPv0ydHulSsnYskSS5ju3Tv9MB0fL+DLLzV47TUPLF6sgk6XwRMJAnTz50NSq62K5OfOQfX991noPWWX4vx5eB4+DOHhw7zuChEREVGBle0QrVKpsGnTJqxevRpTpkzBjz/+iH379qFIkSIp6snlcgwZMgQdOnTI7imJcpTk64vE5cshZfD+fUkQoF2yBMaQkBzu2XNly4pYvFiLkyfj0LevAQpF2mH6yRMZJk1yweuve2DlSmWGJpLFcuWQMHKkzTLNrFkQbt/OQs8pS3Q6uAwcCO/mzVHxo49QtGFDKLZsyeteERERERVIQnR0NDd/JYeg0+kQGRmJ0qVLQ6PR5HV3oP7qK2imTUuzjiSTQbt0KYzdu+dSr2y7eVPA3Lka/PKLEqKY/n3Qr7xixqRJOrRvb4KQRnVdTAw8mzSBy/XrVmXGNm2Q+Msv2ek2ZUR8PNx69YJi374UhyWVCgm//w4zr+7JU472vkUcE0fEMXEsHA/HwzFxPlwqmygV+hEjYGzVKtVySSaD9rvv8jxAA0CZMhIWL7ZsjdWhgzHd+leuyNGvnxuaNnXD338rIKX2pzS1Gjc//thmkXLrVs6G5rToaLh16WIVoAFAMBjgFhIC2bNFHYmIiIgod2Q7RD969Ah79+7Ff//9Z1X2v//9D/Xr10e5cuXQrVs3XLlyJbunI8o9z0KyWLq0VZEkk0G7bBmMwcF50LHUVawo4uefE7F7dzyaNEk/TP/zjwKdOrmhfXs3nDght1knvlYtaHv0sFnmMmECEB+frT6TbcLjx3Dv0AGKo0dTrxMbC7euXSFEReViz4iIiIgKtmyH6O+++w6dOnXC8ePHUxxfsWIFxowZg4iICDx9+hS7du1CUFAQnjx5kt1TEuUaqXBhJK5cCdHb+/kxlQraH36AsWvXPOxZ2mrVMmPjxkT88Uc8Xn89/Rug9+9XoHlzd4SEuOLCBeu3hbjPPkvxM0giu30bmlS2w6KsE+7dg1vbtpCfOZNuXdmdO3Dr1g2IicmFnhERERFRtkP0/v37IZfLERQUlOL4ggULAADDhw/H6tWrUbduXTx48ABLlizJ7imJcpW5Zk3EHzgA3eTJ0I0bh7gTJ2Ds0iWvu5UhjRubsWtXAlatSkClSuZ062/ZokT9+u4YOtQFt249v1laKlIEuhkzbD5GtXQpZBkIe5Qxwq1bcGvdGvKLFzP8GPm//8Ktb1/AYMjBnhERERERYIcQHRkZCV9fX7i7uycfO3fuHCIjI/HWW29h+vTpaNu2Lf73v/9BLpfjzz//zO4piXKdVKIE9GPGQP/pp5D8/fO6O5kiCEBQkAmHDsXj228TUaqUmGZ9URSwZo0Kb7zhgUmTPPDkiQIAYOzZ0+Ye2oIowmXUKMCcfkintMmuXoV7mzaQ21jIDQDMfn6Ir17dZpli7164DBuG1G9wJyIiIiJ7yHaIfvLkCYoXL57i2JEjRwAAbdq0ST7m6+uLcuXK4caNG9k9JRFlgVwOhIQYcfJkHObM0aJo0bTDtMEg4McfXdGxYzV88YUbYmIFaBcsgKRUWtVVnDoF1U8/5VTXCwRZRATc2rSBLJWtw8xly+LJxo24snAhTJUq2ayjWrsW6pkzc7KbRERERAVetkO0TCZD/EsLCx07dgyCIKBOnTopjnt6esLAyw2J8pRaDQwZYsA//8Rh4kQdPDzSnrnUauX46it31Kzpga//rIqE4aNs1tNMnw7h7t2c6HK+Jzt9Gm7t2kF2/77NcnNgIBK2boVYujTMHh54GhoKsUQJm3U18+fzDxpEREREOSjbIdrf3x/Xrl3D06dPAQBGoxG7d++Gi4sLatasmaLu48eP4W1jcSIiyn0eHsD48XqcPh2HYcP0UKvTDtNPnsgwebILqq+ZgqfFKliVC3Fx0HzySU51N9+SHzkC9/btIUtl0UVz9epI2LwZ0guhWSxVCglr10Ly8LD5GM3YsVBs25Yj/SUiIiIq6LIdops2bQqj0YgBAwZg27ZtGD58OJ48eYJmzZpBoVAk14uJicGNGzfg5+eX3VMSkR15e0uYOVOHkyfj0LevATJZ2mH6+l1XdHu41GaZauNGKHbsyIlu5kvyvXvh1rkzhNhYm+Wm2rURHx4OqWhRqzKxWjUkrFoF6YX32SSCKML1vfcgP3nS7n0mIiIiKuiyHaJHjhwJX19f/P333wgJCUFYWBg0Gg0mTJiQot727dshSRLq1q2b3VMSUQ4oVUrCN99ocfRoPDp2TPu2i7/QHKvQ22aZy9ixQGJiTnQxX1Fs3w634GAIqfysTA0bImH9esDLK9U2zE2aQLt4sc0yQauFa/fukF27Zo/uEhEREdEz2Q7RPj4+2L17NwYPHoymTZvi3XffxZ49e1ClSpUU9Q4fPoyqVauiVatW2T0lEeWgV14RsWKFFnv2xKFJE32q9cZgPp6gsNVx2a1bUH/5ZU520ekpN2yAa+/eEPS2f77Gli2RsHYt8MKuB6kx9ugB3eTJNstkjx7BtWtXCI8eZau/RERERPSc9XWAWVCyZEnMmTMnzToLFy60x6mIKJe89pqIX3+NxoYN0fjhh7I4dUqVovwhfDABX+AHDLJ6rGrRYhiDgyFWrpxb3XUaytBQuAwfDkG0vTq6sUMHJP7wA6BS2Sy3RT96NITbt6H+3/+syuTXrsG1Rw8khIcDrq5Z7jcRERERWWR7JpqI8rc33ojDli1PERqagMDAlHtBL8cAHEB9q8fIzCZEtR+Dm9e5Z/GLVD/+CNcPP0w1QBt69EDi8uWZCtAAAEGAbu5cGFO50kdx4gRc33sPMJky22UiIiIieoldQ/SDBw8QFhaGmTNnYsKECZg5cybWrl2Lhw8f2vM0RJTLBAFo29aEgwfjsXRpIkqXtoRACTIMxvcw2riopdKjw1j6RhjGj9fgwQMht7vscFRff225XzwV+vfeg3bJEsDGQmEZolAg8aefYKpVy2axcvt2aCZMACT+YYOIiIgoO+xyObder8fkyZPx888/w2g0WpUrlUr0798f06ZNg1qttscpiSgPyOVAz55GdO5sxIoVKsydq0bEoyqYh7H4BNa3dMwyf4zAZR0RGuqDoUP1GD5cj0KF8qDjeUmSoJ41C5q5c1Otoh8+HLrp0y1/rcgONzckhoXBrWVLyK9ftypWL18OqVQp6EfZ3uubiIiIiNKX7ZloURTRs2dP/PjjjzAYDChatCjq1q2LTp06oW7duihatCgMBgOWLVuGkJAQSJwFIXJ6ajUweLABp0/H4dNPdfjaYxKuoaxVvcKIxnyMQUKCgHnzNHjtNQ8sWKDGxYuygjEhKknQfPppmgFa9/HH9gnQSacsVgyJv/8O0dvbZrlm2jQow8Lsci4iIiKigijbIXr16tX4+++/4eHhgW+++QYRERHYsmULli9fji1btuDChQtYtGgRChUqhN27dyM0NNQe/SYiB+DuDowbp8fhM2b8GbTQZp3eCEVz7AQAPH0qw/TpGtSp44EKFTwQEuKKb79V4fRpWf67XVcUoRk1CuolS1Ktop0xA/qPP7ZbgE4+dblySAwLg+TiYrPcZdgwyPfutes5iYiIiAqKbIfosLAwCIKAlStXok+fPlC8dD+fXC5H7969sWLFCkiShDVr1mT3lETkYIoUkdBrVWPEvNPJZvkSfAA1dCmOPX4sw5YtSnz6qQuaNPFA2bKe6NLFFfPnq3H4sByp7P7kHEwmuAwZAvWKFalW0c6fD8Pw4TnWBfMbbyBx+XJIMuu3ecFohFufPpCdP59j5yciIiLKr7Idov/991+UKVMGjRs3TrNe48aNERAQgH///Te7pyQiByUsnA3J09Pq+Cv4DxMxK83HxsUJ+OsvJWbM0KB1a3f4+3uibVs3zJypxt9/KxAfn1O9tjO9Hq79+0O1dq3NYkkmQ+LSpTAMGJDjXTG1aQNdKpeSC7GxcAsOhnD7do73g4iIiCg/yXaI1mq1KFy4cIbqFi5cGDqdLv2KROSUpOLFofvsM5tlH2MOAnEhw23p9QIOHlRg3jwNOnVyQ5kynmjWzA2TJmmwdasCT5864IrfWi1cQ0Kg3LTJZrGkUCDxf/+DsWfPXOuSYcAA6FJZSEwWFQW3bt2A6Ohc6w8RERGRs8t2iPb19cWVK1eg1WrTrJeYmIgrV67Ax8cnu6ckIgdm6N8fptdftzqughFLMRRA1lYUM5sFnDypwOLFavTq5YayZT1Rr547xo7VYP16Je7ezeNQHRcHt65dody1y2axpFYjMTQUpg4dcrljgH7yZBiCg22WyS9cgFvv3nDu6+eJiIiIck+2Q3TDhg2RkJCAiRMnpllv4sSJSEhIQKNGjbJ7SnpJeHg4OnbsiICAAHh5eeHmzZt53SUqyORyaL/6CpJcblXUBHtxaNAyfPihHjVrmiCTZW+J7ogIOX78UY333nPFq696olYtdwwb5oLQUCVu3BBybwXw6Gi4deoExcGDNoslNzckrF0LU6tWudShl8hk0C5eDFMq77+KAwfg8uGHgCjmcseIiIiInE+294keMWIE1q1bh59//hnHjx/HkCFDULlyZfj4+ODBgweIiIjA0qVLceHCBahUKnz00Uf26De9IDExEfXq1UObNm0wfvz4vO4OEcTq1WEYOhTqxYutyt5aNxFVjreA9Lk3YmOB48cVOHxYjoMHFTh5Ug6DIeszyteuyXHtmhyrV6sAACVLiqhXz4S6dc2oV8+ESpVE2FhnK1uEhw/h1qkT5Kks0iV5eiJh3TqYa9e274kzS6VCwqpVcG/dGvKICOvidesg+flBN21aHnSOiIiIyHlkO0RXrFgR3333HT744AP8+++/NkOyJEnQaDRYunQpKlasmN1T0kt69OgBAIiw8YsxUV7RffwxlBs3QvbSwlWyJ0/g2qMHDAMGwPOdd9CsmReaNTMB0EOnA06elOPwYQUOHZLj6FEFEhKyHqqjomRYt06Fdess3xcpIqJuXTPq1jWhfn0zqlUzQ5GNd0EhKgpuHTtCfvmyzXLR2xsJ69dDrFEj6yexp0KFkPDbb3Bv0QKyqCirYvXXX0P084Nh0KA86BwRERGRc7DLnEynTp2wb98+hISEwMfHB5IkJX/4+PigT58+2LdvHzp27Jil9nU6HSZOnIjWrVsjMDAQvr6+qFixIlq1aoXVq1fDaDTa42lkSlhYGEaOHIkmTZrAx8cHXl5e6e6BferUKXTr1g3+/v4oWbIkmjdvjg0bNuRSj4lymbs7tF9+abNIcfw4XIcMgecrr8A1OBjK1ashPH0KjQaoX9+MsWP1WL8+ETdvxmL37njMnKlFmzZGFC6cvcuNnzyxbKs1aZIL3n7bHQEBz7fVOnRIjsyseyjcuGGZ1U0tQBcvjoQtWxwnQD8j+fkh4bffbK6iDgCaCROgSGVhNCIiIiKyw0x0kldeeQWLn126GRsbi/j4eLi7u8PzhV/UGjdujJiYGJw+fTpTbSckJOCnn35CrVq10LJlSxQtWhTR0dHYuXMnhg0bhvXr12PdunWQ2fs6zTTMnDkTkZGR8Pb2hq+vLyIjI9Osv2/fPnTp0gUajQadO3eGu7s7wsPD0b9/f9y+fRvDc3C/WKK8YmrTBsa2baHcssVmuWA0QrljB5Q7dkAaORKmRo1g7NABprZtIRUtCoUCqFXLjFq1zBg2zABRBC5elOHQoeeXgN+7l/XXfXy8ZVutv/5SAgDUagm1aplRp44JtWubUbu2Gd7e1jdWy65cgVvHjpDduWOzXbFUKSSEh0MsVy7LfctJYpUqSFi9Gm5dukB46Y+QgiTBdeBAJISH5/0l6EREREQOyG4h+kWenp4pwnOS27dv4+nTp5lur3Dhwrh16xZUKlWK4yaTCR07dsTu3buxc+dOtEpn0Z5ff/0V9erVg7+/v81ys9mMpUuXYtCgQVbnetmiRYtQrlw5+Pv746uvvsK0NO4jNJlMGDFiBGQyGbZs2YLq1asDAMaPH49mzZphxowZ6NChQ4p+TZ06FQsXLkyzD9HcloacgPaLL6DYuxdCOhs9CyYTlLt3Q7l7N6RRo2Bu0ADGDh1gbNcOkq8vAEAmAypXFlG5sgHvvw9IEnDjhgwHDz6/BPz6desFzTJKrxdw+LAChw8/f2ssX94Spt96y4Q33zSjivks3Lp0guzhQ5ttmMuXR8LGjZBKl85yP3KDuVEjaJcsgevAgVZlgk4H1x49kLBjB8QKFfKgd0RERESOK0dCtL3JZDKboVahUKBdu3Y4cOAArl27lmYbd+7cwYgRI+Dr64vNmzdbBWlRFDF06FCsXbsWKpUKg9K5J7BJkyYZ7v++fftw/fp1hISEJAdoAChUqBBGjx6NDz74AGvWrMGECROSy4YNG4ZevXpl+BxEjkoqVQraRYvgMmiQ1axnagRRhGLfPij27YNm7FiY69SxBOqgIEh+fs/rCUDZsiLKlhXRu7el7aiopCAsx6FDCkREZD1UA8DVq3JcvSrHmjUqvIHj2IEgyGD7j4HmypWRsGFDcuh3dMZu3aCNioLLlClWZbInT+DWpQvid+6ExK0JiYiIiJI5RYhOjSiK+OuvvwAAlStXTrOun58ffvzxR/Tv3x9BQUHYvHkzSj+bKRJFEUOGDMHatWvRo0cPvP/++3bt54EDBwAATZs2tSpr1qwZAODgS1vjFC1aFEWLFrVrP4jyirFTJ5hq1oRqzRoow8Mhv3Ahw48VJAmKw4ehOHwYLh9/DFPt2jC2bw9j+/aQbFxVUrKkhC5djOjSxRKqnzwRcPjw85nqM2fkMJszv1hZQ+zDZrSDJ+Jslt/wroXd727Ea/FeKOcjQsjjbaszyvDRR5Ddvg31Dz9Ylclu3oRr9+5I2LwZcHPLg94REREROR6nCtEGgwHz58+HJEl4+vQp9u7di8uXLyMkJASNGzdO9/FBQUFYvnw5BgwYgHbt2mHz5s3w8/NLnoEODg7GkiVL7H5v9dWrVwEA5cuXtyrz9fWFu7t7ujPpaXn69CkiIyNx/fp1AMClS5cQExOD0qVLo3Dhwllul8iepIAA6D/5BPpPPoHs0iUow8Oh/OOPVLeGSo3i2DEojh2Dy6RJMNWqZbmHun17iGXL2qxfpIiEtm1NaNvWBACIi7Nsq3XokGWm+uRJOfT6tBNvC+zARnSEK7Q2yw+gPto+3oLY8YUAAN7eYvI91bVrm1Czphmurpl6mrlHEKCbMweyO3eg3LrVqljxzz9wfe89JIaGIltLmRMRERHlE071G5HBYMAXX3yR/L0gCBg+fDim2LgUMTUdOnSA2WzGwIEDERQUhJo1a2LDhg3o0qULli5dmiOLk8XGxgKAzfvEAcDDwyO5TlZs3boVH374YfL3wcHBAIBvv/0WISEhWW6XKKeIlSpBP24c9OPGQXb1KpTh4VD88QcUmVx0UHHqFBSnTgFTpsBcvbrlku/27SG+8kqqj/HwAJo2NaFp0+fbap069Xym+vhxBWJjn4fq9vgDaxEMNQw229uJ5uiIjUjE85nax49l2LZNhm3bLAuWKRQSqlc34803zXjrLUuwLlXKesGyPCOXI/HHH+HWoQMUx49bFSv//BMuY8ZAu3AhnGaKnYiIiCiHOFWIdnd3R3R0NERRxN27d7F9+3ZMnz4dx48fx9q1a1MNqS/r3LkzTCYTBg0ahBs3bqBt27ZYtmwZ5PLs3TuZV0JCQhiWyWmJ5ctDP2oU9KNGQbhxA8pNmyyh2kaYS4v87FnIz56FZsYMmCtXtlzy3aEDxMDANIOfRgPUq2dGvXpmjBkDiCJw6ZIMx47JIQv7HUMO9YcCZpuPDUcQgrEWemjS7JvJJODUKQVOnVLg++8tx0qWFFG7tunZomWWPavTWc8wZ7m6IvHXX+HWsiXkz66eeZHq558hli4N/dixedA5IiIiIseRe3tC2ZFMJoOfnx8GDBiAr7/+GkeOHMH8+fMz/HhJkrBv377k7y9evIgHDx7kRFcBPJ+BTm22OS4uLsN/ACDKz6SAABiGD0fCzp2IPX8e2tmzYapbF1ImZz/lERHQzJkDj7p14f7WW1DPnAnZuXOW5bzTIZMBr74q4n1hOT483C/VAP0ruqMLfk83QKcmKkqGjRtVmDjRBc2aucPf3xOtW7thyhQNtmxR4OHD3J/xlby9kbhuHcRU1mPQzJwJ5S+/5HKviIiIiBxLpmeiX7ycOrO0Wtv3E2bH22+/DeD54l3pkSQJI0aMwOrVq9G5c2e88847GDp0KIKCgrBp0yaUKFHC7n1Muhf66tWreO2111KU3b9/H/Hx8ahVq5bdz0vkzKRSpWAYOhSGoUMh3L0L5ebNlnuoDx2CIIoZbkd++TLk8+ZBM28ezOXKWS757tABYo0aqc5Qq5Yuhcsnn6Tapq5nCEq8vxifnzDh+HEJR48qEBmZvb9J6nQvbq+lBgCULWtOnql+800TcmPbabFsWSSuXQu3du0gJCZalbt89BGk4sVhsrFQIhEREVFBkOkQPWfOHAhZvCdOkqQsPzY19+7dAwAolcoMnX/kyJFYuXIlOnXqhB9++AFyuRwymQyDBw9OXrW7ePHidu1j/fr1sWDBAuzevRtdunRJUZa0unj9+vXtek6i/EQqUQKGgQNhGDgQwoMHUG7ZYrmHev9+CGbbM8W2yK9dg/yrr6D56iuI/v7Jl3ybX3/dMgUNQD1/PjQzZqTahn7gQOi/+AKvyQS89roBgwdbjkdFCTh+XI5jxxQ4dkyO06flMBqz9353/bpl3+uwMMv37u5uCAx0QUCAEkWKyFCokAQvLwmFCkkpvk767OGR/LQyxVyrFhJ/+gmuvXpZ/cFCMJng2rcv4rduhfjCln1EREREBUWmQ3S9evXsHoTTc/HiRfj7+8P1peVtExMT8emnnwIAWrRokWYbkiRh9OjR+Pnnn1MEaADo2rUrAKQI0r523Oe1cePGCAgIwLp16zB48ODkvaJjYmKwYMECqFQq9OjRw27nI8rPJB8fGPr3h6F/fwiPH0OxZQuUmzZB8fffEEymDLcju3UL6sWLoV68GKKfH4xBQYAoQr1sWaqP0Y0cCf2UKTZnsEuWlNChgwkdOlj6oNMBZ87IceyYHEePWoL1gwfZm62Oj5fhxAlPnDiRsfoymQRPTwmFCsFmyE7zWON3IMxfANdRI63aFeLj4RYcjPgdO2xuM0ZERESUnwnR0dEOtESsbbNnz8aSJUtQp04d+Pv7w8PDA1FRUdi1axeePHmCunXrYv369XBxcUm1jaioKDRu3Bj169fHjz/+CIWNrVp+++03DBkyBEuWLEH37t3T7NPKlStx+PBhAEBERATOnDmDOnXqoOyzbXbq1q2Lvn37Jtfft28funTpAo1Gg86dO8Pd3R3h4eGIjIzEjBkzMHz48Kz8aPIVnU6HyMhIlC5dGhpN1u4zJftyqjGJjoZy2zYo//gDit27IRhsr6adVbpJk6AfMybLq1NLEnDzppA8U330qAL//iuDKDruatcqlYTZiskYnfi5zfKHxQKxfvQuaEp4PQvhgJeXhDJlxCzNgDsjp3qNFBAcE8fDMXEsHA/HwzFxPk4Rov/55x+sWLECx44dQ1RUFBISEuDp6YkqVaqgS5cu6N27t81Q/LIbN26gVKlSadb977//UKFChXTbGjp0KNasWZNqec+ePbF06dIUx06ePInZs2fj2LFjMBqNqFy5Mj788EN07tw53fMVBHwDcTxOOyaxsVD++aclUO/aBUGny1Zz2lmzYPjgAzt17rm4OMv2WknB+tgxBWJiHC1US1iBd9EPK22W7kUjtMKfKRZY8/ERMWiQAR9+qEcaf9vMF5z2NZKPcUwcD8fEsXA8HA/HxPk4RYimgoFvII4nX4xJfDyUO3dC8ccfUO7YYXOxrNRIggDtV1/B+O67Ode/F4gicOWKDEePPg/Wly/n/dZ7ShiwGe3QEjttlochGD2xBtJLGz74+YmYPFmH4GBjvp2ZzhevkXyGY+J4OCaOhePheDgmzsep9okmIso0d3cYO3WCsVMnaBMTofjrLyjDw6Hcvh1CXFyqD5PkcmiXLoUxODjXuiqTAZUqiahUSUTfvkYAwNOnSQuWyXHkiIBr14CEBCViYwVIUu7MWhuhQleswz40wms4Y1XeHWsRidIYh3kpjt+5I8OQIa747jsTZs7UoUGDjC8CR0REROSoGKKJqOBwdYUpKAimoCBodToo/v4byj/+gHLrVggv7OMuqdVI/PFHmIKC8rCzFoULS2jZ0oSWLU0p/lKtUmkQGwvExAiIjhYQEyOk+PrFYy9+JB1PTMxcAI+DJ9pgK46gDvwRaVU+FvMRidL4BiOsyk6fVqBdO3e0aWPE9Ok6VKiQ8S3KiIiIiBwNQzQRFUwaDUytW8PUujW0BgMUe/dCfuwYIEkw9OsHqXTpvO5hmmQywMsraSGvzN+Vo9cDsbHWgTtlCMdLx3zR8/EWbI5phMKItmrzK4zCbZTCenSxPiGArVuV2LFDgffeM2DCBD28vXk3ERERETkfhmgiIpUKphYtYEpnq7z8RK0GihWTUKxYZoOsP2QHVkPq3NlqBXQZJPwihKCZ5IuDaGDz0SaTgGXL1Pj1VxXGjtVh0CADePsXEREROZN8utQLERHlFLFBA2i/+85mmVrS4y+39ujnsxlypL5vd2ysgM8+c0Ht2h5Yv14JiZPSRERE5CQYoomIKNOMnTtDO2OGzTJ1wlOseBCEWNfi+Fk5AC3xJxQw2qx765YM773nipYt3XD0aN6vRE5ERESUHoZoIiLKEsOwYdAPGZJquWviY/Q1/oQ/8Q7uwxfL8R5aYyuUMFjVPX5cgVat3PHuuy64fp3/NREREZHj4m8qRESUNYIA3eefw9i+fbpVi+Ap3sP/sBVt8QA+WIF+aIdNUEOXot7GjSrUru2OTz/VIDo6h/pNRERElA0M0URElHVyORK//x6munUz/BAvxKAfVmIT2uMBfLAKvdEBG6GBFgBgNAr49ls1atb0wNKlKhisJ66JiIiI8gxDNBERZY+LCxI2boRu/HiI3t6Zeqgn4tAbodiITngAH/yCnuiM3+GCRDx9KsMnn7igTh13hIcruPgYEREROQSGaCIiyj61GvqJExF36RLi//gD+gEDIPr4ZKoJD8SjJ37F7+iKhyiGMASjG9bi3jUd+vZ1Q5s2bjh1iouPERERUd5iiCYiIvtRKGBu3Bi6+fMRd+EC4jdvhn7gQIjFi2eqGTckIhi/YS264yGKYR26oMzh39C+qYSBA11w65aQQ0+AiIiIKG2KvO4AERHlU3I5zA0awNygAXRffAH50aNQ/vEHlOHhkEVFZbgZV2jRBevRBeuhgxrbf3sHCzZ0QYlBLTFkggaFCuXgc8hpkgQhOhrCzZuQ3boF2a1bgMkEU/PmEKtWzeveERERkQ0M0URElPNkMpjr1oW5bl3oZs2C/MQJS6D+4w/Ibt/OcDMa6NERf6Cj6Q/ol6iw54cWEDt3wFuzWkLu7ZVz/c8qSQJiYiB7ISTLbt16/n1kJIS4OOvHTZ0K/XvvQTd1KuDpmevdJiIiotQxRBMRUe6SyWCuXRvm2rWhmzkT8lOnLIF640bLTGwGqWFAK+MWIGwLDGFKPHrtbXgNCIKpXTtIhQvn4BNISR4fD8X581Dcu5cyKD/7EGJjs9Su+qefoPzzT2gXLICpVSs795qIiIiyiiGaiIjyjiDA/PrrML/+OnTTpkF25kxyoJZfv57hZlQwouTpHcDwHRBHjoK5cSMYO3aEqW1bSJlcMdxKbKz1DPKzD49bt+AbE5O99tMgu3MHbt27w9CtG3SzZ0MqWjTHzkVEREQZwxBNRESOQRAgvvYa9K+9Bv1nn0F27hyU4eGWQP3ffxluRmY2QbZ7N5S7d0MaNQqmhg1h6tABxnbtIBUrZv2AuDjrGeRnYVm4dQuy6Gj7PccsUv32GxS7d0P3xRcwdukCCFxYjYiIKK8wRBMRkeMRBIjVq0NfvTr0n34KWUQElH/8AXHtH3C9cSnjzZjNUO7ZA+WePdCMGQNz/fowV60K2Z07loB86xZkT57k4BOxH9njx3B9/30Yf/sN2gULIPn55XWXiIiICiSGaCIicmyCALFKFeirVAEmTkTcxYuI/Cocio3hqKg/n/FmRBGK/fuh2L8/BzubMaKPD0R//+QPKemzpyc0n38Oxd69qT5W+eefUNSpA920aTC8+y4g426VREREuYkhmoiInIoYGAi/7wNh+nY8Vi24hjtfb8Y7Cb/jNZzJ664lE4sWhVimTIqAnPxRujTg6prqYxM2boRy9Wq4fPppqouSCXFxcBk9Gsp166BdtAhi+fI59VSIiIjoJQzRRETklBQKoP34cogb+hG+/noc+iy6hXb639EV6/A6TuXouUVvb4j+/jD5+SG6cGG4Vq4Mefnyz0Oym1vWGxcEGPv0gal5c7iMHQvlli2pVlUcOgT3+vWh++QTGD780PJDISIiohzF/22JiMipeXgAkybp0b9/ccyYMQZvhn2MAOk6umIdumIdauN4ptuMVnjjoVsZxBQug8Ri/jCWKgOhrD/UlfzhUbUUvMu4wt0d0Ol0uB0ZidKlS0Oj0dj1eUklSiBx9WoowsPhMnYsZA8f2qwn6HRwmTIFqvXrkbh4McRq1ezaDyIiIkqJIZqIiPIFPz8J332nxZAhekye7I+5+8djLsajDG6gCywz1G/hKGSQ8BhFcAMBVh/XURY3UQbxJg8gBpaPG4CtHO7qKqFYMTd4erqhVCkFSpQQUKyYBB8fCcWKifD1ff61u3sWn5QgwNShA+IbNYJm4kSo1qxJtar8zBm4v/029CNHQj92LGDnUE9EREQWDNFERJSvvPaaiPDwBGzfrsBnn2lw5UoAFmAMFmAM5DBBAx0SkNVU+1xiooCbNxUA3HHuXNp1XV0l+PiIz0L186+TQraPjwRf39QDt1S4MLRLl8LYtStcRo6ELDLS5nkEkwmaefOgDA+H9ptvYK5TJ9vPk4iIiFJiiCYionxHEIDWrU1o3jweK1aoMGeOGo8fy2CGwi4BOrMSEwXcuCHHjRvp13V3l1CpkhlVq4qoVs2MqlXNqFLFDA8PwNSsGeIOH4Zmxgyoli2DIEk225Bfvgy31q1hGDgQus8+Q9anwomIiOhlDNFERJRvKZXAwIEGBAcbsGiRGhs2KHHjhgxms5DXXUtVfLyAkycVOHky5fGAAEuwrlpVjaoN56N23S4oN2sY5Jcv22xHkCSoly2DcutWaL/+GqZmzXKh90RERPkfQzQREeV7hQpZFh+bNEkPUQSePBHw4EHShyzF54cPn3/96JHgMIHbMpMtx+bNymdHWqCYx2l84TcTfaO+hFwy2Xyc7PZtuHXpAkOPHtDNmgWpSJHc6zQ5B7MZshs3IIuIgPziRQgPHsBcrRqMwcG8t56IyAaGaCIiKlBkMqBoUQlFi0qoXBkAzKnWTS1wP3wow/37Au7fl3DnjhnR0c8uF8/lwP0wzgXvxX2OhQjGcgzAGziZal3Vr79C8ddf0M6dC1OHDpZr3qlgkSQIt29DfuECZBcuQP4sNMsuXYKg01lVF7/8ErrJk2Hs1s3ywqF8QzDZ/qMbEWUMQzQREVEq0gvcOp0Okc+2uFKpNGkG7qQZ7ocPLV/bM3CfRQ3UwRGMwleYjs/gAutABACyhw/h9u67uFYjCA+nzEO5Br5QqezWDXIUkgThwQNLUH72IbtwwTLLHBeX4WZkt2/DdfBgmJcsgXb6dJgbN87BTlOOkyQow8Lg+s038Ll4EaaqVWEcORLGjh35RxKiTGKIJiIisoOsznA/fCjg/v3nl5T/958M58/LcetW5n6pNUOBeRiHjeiIHzAQTbA31brlzmxCkc77MF42DwcrvYtq1UVUrWp+tpCZiKJFbS9YRo5HePo0OSy/+Fn25IndziE/cwbuHTrA2LIldNOmQXz1Vbu1TblDduUKXMaMgWLfvuRjyrNnoXzvPZgWLYJu2jSYGzXKwx4SOReGaCIiolz2YuC2sA7cMTHAv//Kcf580ocMERFy6HRpz2D/h1fQFLsxED9gLsbBE7ZnHr0Qg2XiQPx1YQ0GXvgBYWHlksuKF7eEasuH5esKFUQo+FtD3omLg/zSJevAfO9ernVBuWMHFLt2wdi7N3QTJ0IqXjzXzk1ZpNNBvWAB1AsXQjAYbFZR/PMP3Nu3h7FFC+imToVYpUoud5LI+QjR0dH8czM5hBcvi9RwIROHwDFxLBwPx5PbY2I2A1evypJD9fnzcpw7J8fdu7Znrf1wG0sxFEHYnGa7iXDBp/gc3+AjiJDbrKPRSAgMtITqevVMaN/e6JA7Zzn960Sng+zy5RRBWX7hAmS3buV1z1KQXF2hHzYM+o8+SncLNacfEycl37MHLmPGQH71aoYfIwkCjD16QPfpp5BKlcq5zlEKfI04H4Zochh8A3E8HBPHwvFwPI4yJo8fCzh/XoZz557PXF+6JIPRKACQ0B1hWIThKIZHabZzBG/hffyIf1E13XN6eEjo1s2Afv0MqFFDtNMzyT5HGZN0GY2QXb1qfRn2tWsQxNz5eUpyOcQKFWB+9VWIgYGWz5UrQ/bvv9BMnQp5BjY2F318oP/kExj69EFqlyo4zZjkE8KDB9BMmgTV2rVZbkNSq2EYMgS6UaMALy/7dY5s4mvE+TBEk8PgG4jj4Zg4Fo6H43HkMTEYgMuXZcmh+tapJ+h9ciy66X9J+3FQ4nN8itn4BEZkbNWxWrVMePddAzp3zvvZ6WyNiSQBRiOg01kufdXrIej1wLOPpK+tygwGy+rWBoPl2AuPT673Qh3ZnTuQXbkCwWjMmR/Cy09LECAGBFiCcuXKEF991RKYK1QA1GrbDzIYoFq+HOovv4Ts6dN0z2GuVAm6qVNheucdq5XfHfl1kq+IIlQ//wzN1KkQYmLSrCoJAgQp/QggenlBP2YMDAMHcruzHMTXiPNhiCaHwTcQx8MxcSwcD8fjbGMiSUDcrzvgPWkUXB/fSbPuOVTFACzHcdTOcPtJs9PvvmtA9eq5MJuq1UJ27x6EqCjI7t6FcPcuxNu3kfjgAdyVSsjN5pTh9sUQbCMoC3p9zvc5h4l+fpaAnBSUX30V5ooVATe3rDUYHQ3NggVQff99hn4+pvr1oZsxA+ZatZKPOdvrxBnJzp+Hy6hRUBw/nmY9SSZD4oABuBIcjPKHDsFt4cIM/ZFELF0aukmTuN1ZDuFrxPkwRJPD4BuI4+GYOBaOh+Nx2jGJjYVm2jSoly9Ps5oZMizESHyG6UhE5kLY66+b0K+fAV26GDOf38xmCA8fWoJxVJQlKN+9mxyUkz9HR2ey4fxDLFbMEpBfnF2uVCnHLr0Vbt2CZubMDF8ibOjSBbrJkyEFBDjv68QZJCRAM2cOVEuWQDCnviMAAJhq1oT2q6+QGBj4fDx0Oqi//hrqpUtt7hX+MnO1atBNmwZT06b2egYEJ/6/pABjiCaHwTcQx8MxcSwcD8fj7GMiP3AALiNGpLvw0GOvsvjq1e8QercZbtywvfBYajw9JQQHW+6drlbVDMTGQvZyIL53D7KoKMv39+5BuH8/3UBQUEienjBXrvx8VjkwEOKrr0IqVixP+iM7fRoun32WYquk1EgqFQwDByJm2DDcio932teJo1Js2waXceMgu307zXqShwd0kyfDMGAAIJfbfN8S7tyBZtYsKNesydA9+cYmTSzbndWoYZfnUtA5+/8lBRFDNDkMvoE4Ho6JY+F4OJ58MSZaLdRffAH1okXpBldD37441m0mlv/ug3XrVIiPf37vqwp6lMBd+OEOSiIqxeekr0vL7sBFTMzpZ+SUJFfX5IBsDgyE+Cw4SyVKWN1jnOckCYqdO6GZMgXyCxfSrS56eeFOv35QjR4NTaFCudDB/E24fRsuH38M5ea0V90HAEOnTtDNmmX5d/RMWu9bsogIaKZNg/LPPzPUF0NwsGUl7zJlMvckKIV88X9JAcMQTQ6DbyCOh2PiWDgejic/jYns9Gm4DhsG+fnzadYTixeHoVcvmO4+wqPT96C/fh9FdHfSXfm7IJIEwbJwl1oNSa0GVCpIGg3g5gZzxYrJ9y2bX30Vkr+/891rajJB+csv0MyalaH9qs2lS0M/ZQqMnTs733N1BCYTVN9/D83s2RDi49OsKpYpA+38+TA1b25VlpH3Lfn+/dBMmQLFqVPpdktSqWB4/33ox46FVKRIxp4LpZCf/i8pKBiiyWHwDcTxcEwcC8fD8eS7MTEaof7mG6i/+MKy6JaTEtVqy0rCLwZXlcoSZF/8+sVw++x7qNWQVCpAo7F8VquTH59cP6nOC99blWk0li2fHG0WOSckJEC9eDHU33wDISEh3eqmmjUti481aJALncsf5CdPwmXkSMjPnUuznqRUQv/RR9CPGQO4utqsk+H3LUmCcuNGqKdPh/z69XT7KHl6Qjd6NAyDBwMuLunWp+fy3f8lBQBDNDkMvoE4Ho6JY+F4OJ78Oiayy5fhMnw4FEeP5nVXUpDUakjFi0MsWRJiiRKWr0uUgFSyJMTixSGVLInEQoUQ+fhxvhsTZyDcvw/1nDlQrVyZoXvaje+8Y7mvtlKlXOidk4qJgWbGDKiWL093SypT3brQfvUVxMDANOtl+n3LYIBqxQrLdmeP0r/iRPTzg+6TT2Ds2ROQZ24NhYIqv/5fkp8xRJPD4BuI4+GYOBaOh+PJ12MiilD9+CM006ZlaHYxW6eCgAfweXb3tB9MxYrD763iqNSkGJQBzwJziRKQChdOd2Y3X4+Jk5BdugTNlClQbt+ebl1JLoehb1/oP/4Ykq9vLvTOSUgSlOvXQzNxImT376dZVSxSBLrp02EMCcnQlQ9Zfo3ExkK9aBHU334LITH9tQ3MlStDN2UKTC1bFowrMrKB71vOhyGaHAbfQBwPx8SxcDwcT0EYE+HWLbiMGgXlX39l6fGiuweeupbElQQ/XE6whOQolEwOzFEoiXsoDhOUVo/19JTQvbsB/fsbULlyxvadLghj4izk+/dD89lnUPzzT7p1JTc3y2XIw4ZlfU/rfEJ2/To0Y8ZAuXt3unUNvXpBN2MGJG/vDLef3deIcO+e5YqDVasydMWBqUED6KZPT7F3OKXE9y3nwxBNDoNvII6HY+JYOB6Op8CMiSRBGRYGzdSpyQtISQrF88upS5SA+OxDevFz8eKAh0dSE/jnHzlWrFDh99+VSEjI3MzUW29Z9p3u1MmY5u2WBWZMnIUoAmFh0MyYAXVUVPrVixeHbuJEy6xqQbsU2GCwrEkwb166ezabK1WCdv78LN1Xbq/XiOzyZctK3lu2ZKi+oVMn6D/7DGLZslk+Z37F9y3nwxBNDoNvII6HY+JYOB6Op8CNidEI4c4dwM3NMvOVxRWWY2OB335T4X//U+H8+cwFpUKFns9Ov/qq9ex0gRsTJ6DT6XD76lVU3LED7gsXQoiJSfcx5ldfhW7aNJhatCgQlwLLDxyAy5gxkF+6lGY9SaOBfuxY6D/6yLLYXRbY+zUiP3LEcsXBsWPp1pUUChj694d+/Pg82+vcEfF9y/lwfwEiIiLKGKUSUkCA5ZffbGxR5OkJDBhgwP798fjrr3j06WOAq2vG/qYfEyNg2TI16tb1QKtWblizRgmtNstdoVwiqVRIHDoUcadPQ//hh5YVzNMgv3ABbsHBcOvQAbLTp3Onk3lAePwYLh98APd27dIN0MZmzRB/+DD0Y8dmOUDnBHOdOkj4808krFoF8yuvpFlXMJmg/uEHeNSqBfXcuUAOr7dAlFMYoomIiChPCALw+utmLFqkxYULsZg3T4sqVdK/xzLJ0aMKDB3qisBAT0yYoMHFi/y1xtFJhQtD9/nniDt2DIYuXdKtr9i3Dx5NmsBl0CAIt27lQg9ziSRBuWoV3N94A6pffkmzqujri8SffkLiunWOeym0IMAUFIT4w4ctK4Sns0icEBcHzeefw+P116FcsQIwmXKnn0R2wv9tiIiIKM8VKgS8/74BBw7EY9euePTunbnZ6e+/V6NOHQ906FAYW7cWQXx8/r8E2JlJAQHQLl+O+N27YapXL936qrVr4fHmm9B89hkQHZ3zHcxBsosX4damDVyHD4fs6dNU60mCAP3AgYg7dgzGzp2d47L2Z5drx508Cd3EiZDc3dOsLrt3D64jR8K9Xj0otmyxLJ5A5AQYoomIiMhhCALwxhtmLF6c1dlpFaZMKYcqVYqhRw9XhIYq8eSJE4SPAspcqxYStmxBwi+/wFyxYpp1Bb0e6m++gUfNmlB9/TXk+/ZBdvUqkIHtlhxCYiLUM2bAvWFDKA4fTrOquVo1JOzaBd3cuZa/MDkbd3fox49H3D//QD9wICSFIs3q8suX4RYSArfWrSHPwL3VRHmNC4uRw+CiCo6HY+JYOB6Oh2OSOyQJOHlSjv/9T4X165XQajMXiuVyCQ0amBEUZETbtkaUKMFffXJThl8nJhNUK1dCPXs2ZA8fZuocopcXpJIlIZYsaVkZvmRJiH5+yV9LJUtC8vLKs9lcxa5dcBkzBrKbN9OsJ7m7QzdxIgyDBgHpBM+syov3LdnVq9BMnw7lH39kqL6xXTvopkyBmM491vkF/y9xPgzR5DD4BuJ4OCaOhePheDgmuS86+vnK3hERWdsCqXZtE4KCjAgKMiIggL8G5bRMv07i4ixbPX37LQQ7zjJLLi6W7deehe0XA7fk52cp8/Gx69Zawt270EycCNWGDenWNQYFQTtnDiQ/P7ud35a8fN+SnzhhWcn70KF060pyOUzNm8PYvTuMrVsjzb3tnBz/L3E+DNHkMPgG4ng4Jo6F4+F4OCZ5R5KAEycss9MbNmR+djpJ1arm5ED96quiU9x26myy+joR7t6FZtYsKENDIYjW25nlBEkut+x//uKM9rOZbLFECcvsdvHiQHrPw2yGavlyaGbOhBAbm2ZVsVQpaOfNg+mdd+z4TFKX5+9bkgTFn39CM20a5BcuZOwhHh4wBgXB0L27ZW/sfLaHeJ6PCWUaQzQ5DL6BOB6OiWPheDgejoljiI4G1q5VYcWKrM9OA0CFCkmB2oSaNc0M1HaS3deJLCICmilToNy5Mwd6lzWit7ftGe2SJSHJ5dBMmwbFqVNptiHJ5TAMGwbd+PGAm1su9dyB3rfMZih/+QWa2bMhi4rK8MPEkiVh7NoVhu7dIVapkoMdzD0OMyaUYQzR5DD4BuJ4OCaOhePheDgmjkWSgNOnTVizRocDB3wQEaHMclt+fiLatbPMUNeta85vE1+5yl6vE/nevdDMnAnF8eN27F3eMNWubdkKKg9CoMO9byUmQv3dd1AvXJjurP3LzFWqwNC9O4xdu0IqWTKHOpjzHG5MKF0M0eQw+AbieDgmjoXj4Xg4Jo7nxTGJinLF5s0KbNqkxPHjWV+kqWhREW3aWO6jbtTIBLXajh0uAOz9OhHu3YPsxg3I7t6FcOeO5XNUFGR370J25w6Eu3chOOi+w1KhQtBOmwZj376ALG82yXHU9y3h8WOo582D6scfIRiNmXqsJAgwN2oEQ3AwjEFBgKdnDvUyZzjqmFDqGKLJYfANxPFwTBwLx8PxcEwcT2pjEhUlYMsWJcLDlTh4UA5RzNq12p6eElq1MqJdOyOaNzfl5lW4TivXXyeiCOHRI0uwfvYhPAvYyYE7KgpCQkLO9+UFhuBg6GbOtCxelocc/X1LuH0bqjVroAwLg/y//zL9eMnFBcY2bWAMDoapaVNAmfUrUnKLo48JWWOIJofBNxDHwzFxLBwPx8MxcTwZGZPHjwVs3arA5s1K/P23AgZD1gK1RiOhWTPLDPU77xjh5ZWNjudjDvk6kSQgNtYyex0V9TxYP/s+aUZb9vhxtk9lLl8e2gULYG7c2A4dzz6HHA9bJAnyf/6BMiwMyt9/h+zRo0w3IXp7w9i5M4w9esBcq1aebXGWHqcZE0qWMxvQERERETkob28JffoY0aePEbGxwM6dSmzapMDOnUokJGT8l2ydzjK7vWWLEgqFhEaNTAgKMqFNGyN8fTlH4dAEAShUCGKhQhADA1Ovp9NZLh9/aRb7xcAt3LsHwWy2eqikUkE/ahT0o0alv5o3WRMEmGvVgrlWLehmzoRizx5LoN6yBYJWm6EmZI8fQ/3DD1D/8APM5cvDGBwMQ/fukAICcrbvlO9xJpocBv8K53g4Jo6F4+F4OCaOJztjotUCf/9tuYd62zYFoqOzds+qIEioU8ecvDCZv3/B/lUr379OzGYIDx48v0c7KgpQKGBs1QpSqVJ53TsrTj8ecXFQbt4M5dq1UOzdm6Xtz0xvvQVjcDCMnTpBKlIkBzqZOU4/JgUQQzQ5DL6BOB6OiWPheDgejonjsdeYGI3AwYNybNqkxObNSty/n/VFoF57zTJDHRRkRMWKubPfsSPh68Sx5KfxEO7ehXLdOqjWroX83LlMP15SKmFq0QKG7t1hatUqz64YyE9jUlDkzbKARERERA5MqQSaNDFj/nwdLlyIw59/xmPYMD38/TMfgk+fVmDGDA1q1/bAW2+5Y8YMNf76S4EHDxzz/kwiZyGVKAHD8OGI378fcYcOQTdyJEQ/vww/XjAaody6FW79+sGzYkW4fPQR5AcOAFmY3aaChTPR5DD4VzjHwzFxLBwPx8MxcTw5PSaSBJw9K0ueob54MXsbSBcvLqJ6dTOqVTOjenUzqlcXUaaMmFe7H+UIvk4cS74fD1GE/MABqNauhTI8PNN7TwOAWKoUDN26wdi9e9r3zNtJvh+TfIgLixERERFlkCAANWqIqFFDj0mT9Lh8WYbNmy0Lk/3zT+Z/rbp3T4Z792TYseP5NjyenhKqVEkK1ZaAHRgoQqWy5zMhyqdkMpgbNYK2USNo586F4s8/ofr1Vyh27crw/uGy27eh+eoraL76Cubq1S37T3ftCql48RzuPDkLhmgiIiKiLKpYUcTo0XqMHq3HrVuW1bo3bVLi8GE5JClrl2vHxgo4fFiBw4ef/5qmUkkIDBRfmLE2o2pVMzw87PVMiPIhFxeYOnaEqWNHCI8fQ7lhA5RhYVAcP57hJuRnz8Ll7FloPvsMpiZNLAuStWsHuLvnYMfJ0TFEExEREdmBv7+EoUMNGDrUgAcPBGzbZlnpe+9eBYzG7N3/bDAIOHtWjrNn5QgNfX68XLnnl4EnBWxur0VkTfL2huH992F4/33Irl2Dcu1aKMPCIL9+PUOPF0QRyt27ody9G5KrK4xt28LUogVM9etDysR92JQ/MEQTERER2ZmPj4R+/Yzo18+I6Ghgxw7LDPWuXQpotfZbUOzaNTmuXZNj48bnx3x9U85YV6smomxZ57rP2mgEYmIEREc//3j61PI5IQEoV05EkyYmeHrmdU/JGYnlykH/8cfQT5gA+YkTlkD9+++QPXmSoccLiYlQ/fYbVL/9BgAwly0Lc/36MD37kPz9c7L75AAYoomIiIhykJcXEBxsRHCwEQYDcPGiLHlW+dw5Oc6flyMuzn7B+v59Ge7fl2HXruf3WXt4SKha1XIJeFK4fvXVnL3POikI378vx+XLbrhyRYWEBGWKYPzyR1Jwjo9P/+dRqJCEIUP0GDpUDy+vnHselI8JAsxvvgnzm29C9/nnUPz1lyVQb90KQa/PcDPy69chv34dqtWrAQBi6dLJgdrUsCGkMmUsCypQvsHVuclhcGVCx8MxcSwcD8fDMXE8zjgmogjcuCHDuXPPw/XZs/Js7U2dEUqlhEqVxBQLmFWrZk4xu5vWjLA9grA9eHhIGDRIjw8+MMDbm7/WpscZXyO5LiYGyvBwy/7TBw5AkLL370r080sO1eb69SGWL58iVHNMnA9DNDkMvoE4Ho6JY+F4OB6OiePJT2Ny/76Ac+eSZqwtAfvq1extqZURpUuLkCTkahC2Bzc3CQMGGDBsmB4+Pvz1NjX56TWSG4Tbt6Fctw6qsDDIL1ywS5ti8eIw1atnuQS8QQMk+vsj8vZtjokTYYgmh8E3dcfDMXEsHA/HwzFxPPl9TOLigH//lae4HPzCBRkMBucJuznNxUVC//4GfPSRHsWL89fcl+X310iOkSTIzp+37D+9cSNkkZF2a9pctChiatSAomlTCE2aQHz1VTjVIgaZodNBFhUFyd0dko9PXvcmyxiiyWHwTd3xcEwcC8fD8XBMHE9BHBODAbh0SZYcqs+etdxnHRtbsIO1Wi2hb18DRozQo1Qp/rqbpCC+RnKCcPMmFAcPQnHwIOQHD0J+44bd2haLFIG5bl2YGjSAqX59iFWrOkeoNpkg3L0L2e3bkN25A+HOneSvk79/9AgAoJsyBfpRo/K4w1nHhcWIiIiInJhKBVSrJqJaNRGAEYDlPutbtwScOWMJ1knh+u5dx/pF3NVVgpdX5j4ePhSwcKEaO3cq02xbrxfwww9qrFihQu/eBowcqUeZMgzTZB9SmTIwlikDY69eACyXfacI1VevZrlt2ZMnkG3ZAuWWLZZzFSoEU926lnuqGzSAuVo1QJHLMU4UITx4YAnDLwfjpK/v3YMgihlqTrh9O4c7nLMYoomIiIjyGZkMCAiQEBBgQocOpuTjDx8m3Wf9fBGzq1dlkKSsz1q7uVnCbaFCtkOvm5sBZvMjlCtXGD4+yhRlWVkdvGJFoH79RJw6JcfcuWps25Z2mDYaBfzvf2qsWqVCjx5GjB6tR7lyGftFnyijpFKlYOzeHcbu3QEAwt27UBw6BHlSsL50KcttCzExUG7fDuX27ZZzeXg8D9X168NcowagTPt1kHbnJQhPn9oOx7dvQ3b7NoS7dyEYjVk/x0tkTh6ieTk3OQxeXuR4OCaOhePheDgmjodjknnx8Snvs759W4CLCzI8M5xeEM7pMTl7VoZ58zQID89YiJDJJHTrZsTYsXq88krBC9N8jeQN4cEDyA8dej5bHRFht7YlNzeY3norea9qc61aSPHCjItLGY4jI63CsqDV2q0/GWGuWhXxBw7k6jntiTPRRERERAWYuzvw1ltmvPWWOa+7kiXVq4tYuTIREREyzJ+vxvr1yjRn1kVRQFiYCmvXKtG5syVMv/pqwQvTlLskHx+YOnaEqWNHAIDw+HHyLLXswAEoIiKyvJWWkJAA5e7dUO7ebTmXiwvM1atDiIuzzCLHxtrradiNcOdOXnchWxiiiYiIiMjpVa4sYvlyLSZM0GPePDXWrVNCFFMP05Ik4PffVfj9dxXatzdi7FgdqldnmKbcIXl7w9S+PUzt20On0yHq339R9vZtuBw7Zrmn+uzZDN9f/DJBq4Xi6FE79zjrRB8fiH5+kPz8IPr5QSxVCpKfHyBJKfbLdiYM0URERESUb1SsKGLZMi0+/liP+fPVCAtTwmRK+xf18HAlwsOVaN3aiPHj9ahZ0zln5cl5mT09oW/VCkKHDpYDMTFQHDnyfKGy06chmB3v36Xo5WUJx6VKpQzKfn6QSpWCWLIkoFbndTftjiGaiIiIiPKdcuVEfPutFuPG6bBwoRqhoSoYjWmH6W3blNi2TYkWLYwYN06P2rUdL7RQAVGoEEytWsHUqpXl+7g4KI4dg/zAAUuwPnUKgsmUdhvZJLm5PQ/EL4fjpIDs7p6jfXBUDNFERERElG8FBEhYuFCHsWP1+PprNVauVEGvTztM79ypxM6dSjRpYpmZrlePYZrymIcHTM2awdSsGfQAkJAA+fHjUCSF6pMnIRgMGW5OUqkglixpdYl10oyyWKoUUKiQ015undMYoomIiIgo3ytVSsLcuTqMHq3HN99Y9o/WatMOCHv2KLFnjxINGpgwbpwOjRqZmSnIMbi5wdykCcxNmlhCtVZrCdUHD0Jx+DCEu3chFS78fBY56XLrpM9Fi1r2wqMsYYgmIiIiogKjRAkJs2frMGqUHosXq7F8uQoJCWkn4wMHFDhwwB116pgwbpweTZuaGKbJsbi4wNyoEcyNGllCNeUo/vmBiIiIiAocHx8J06frcPZsHMaM0cHDI/3thY4cUaBLFzc0b+6G7dsVyOKORETk5BiiiYiIiKjA8vaWMHmyHmfPxmHCBB08PdNPxidPKtCjhxsaN3bHpk0KZHEnIiJyUgzRRERERFTgFS4s4ZNP9Dh3LhaTJulQuHD6yfjsWTn69HFDgwbu2LBBCQfcgYiIcgBDNBERERHRM4UKAWPHWmamp03TomjR9MN0RIQc/fu7om5dd6xdq0QO7zxERHmMIZqIiIiI6CUeHsCIEQacOROHzz/Xwtc3/TB9+bIcgwa54q233DFjhhobNypw9aqMl3sT5TNcnZuIiIiIKBVubsCHHxrw3nsGrFqlwsKFakRFpT0PdfWqHPPny5O/d3eXULWqGdWqWT6qVzcjMFCERpPTvXcOogjcvi3g8mU5Ll2S4eZNGVQqoHhxEcWLSyk+u7vndW+JGKKJiIiIiNLl4gIMGmRAv34G/PKLCgsWqBEZmbGLOuPjBRw5osCRI89/9ZbLJVSqJKJqVUuotgRsEUWK5N8lv41G4Pp1GS5dkuHSJTkuX7Z8vnJFhsTEjO0Z5u4uWYVrX18RJUqk/OzhkcNPhgo0hmgiIiIiogxSq4H+/Q3o3duAX39VYv58NW7ckKf/wJeYzQIiIuSIiJBj7drnx0uVEpNnrJM+ypSRnGpf6sRE4PJlGS5ffh6UL1+W4epVGUym7D2R+HgB//0nx3//pV3P3d0SposXl1CihAhfX+vwXbw4wzZlDUM0EREREVEmKZVAnz5G9OxpxLp1Ssybp8Z//2U+TL/s9m0Zbt+WYds2ZfIxT0/JKlgHBopQqbJ9umx5+lTApUuyFEH50iU5IiMFSFLepv74eAHx8XJcvZp2PTc32+H65RluDw841R8yKGcxRBMRERERZZFCAfToYUS3bkZs3KjEypUqnDolR1yc/RJXbKyAgwcVOHjw+a/uSqWEwEAxxX3WVauaUaiQ3U4LAJAk4O5dIUVQvnjR8vnhQ+dfozghQcDVqxkL2y/ObPv7i2jWzIR69cyQOf+PgTKJIZqIiIiIKJvkcqBLFyO6dDFCFIGbN2U4e1aGc+fkyR/pLUiWGUajkNzui8qUEZNDdVLA9vNL/3Jws9nS54sXU84sX7kiR2wsp2ATEgRcuybHtWvPj331leXy++BgA4KDjQgM5DLsBQVDNBERERGRHclkQNmyIsqWFdGhw/NNox89EnD+/PNgffasJaiKov1C6s2bltWtN29+fjl4kSIiqlUTn10GDri6uuHkSTWuX1cnB+arV2XQ63M3LCuVEsqXF1GpkoiKFc3JnytUsITR+/dluHtXsPn53j0B9+7JEBOTtwH/9m0ZFizQYMECDWrUMCM42ICuXY3w9c2/C8QRQzQRERERUa4oWlRCkyZmNGliTj6m1QIXLshx7pwMZ89awvX58/IMr1adEU+eyLB3rwx79yoAqAHY+ZrvdLi5SahY0YyKFVMG5oAAEUpl6o+z/CECAMyp1klMRIpQ/fLn+/cF3L2bO2H7zBk5zpxxweTJGjRtakJwsBFt2xrh5pbjp6ZcxhBNRERERJRHXFyAWrXMqFXLDMAIwHJp9fXrSaFaljxr/eCBY9986+0tPgvKzwNzpUoZu5w8q1xdMxa2tVokB2pbn5OCd3R09n/Goihg1y4ldu1Sws1NQrt2RnTvbkTjxibIs7/2HDkAhmgiIiIiIgcilwMVKoioUEFE587Pj9+/L7xwj7UlZF+9Ksv1lbBLlRJTzCxXqmSZWfb2dtxLmF1cgIAACQEBqQdt4HnYtjWrfe+e5VLyyEgZ4uMz9jNPSBAQFqZCWJgKxYuL6NLFiO7dDahWTeRq306MIZqIiIiIyAn4+krw9TWhefPn91nHxwMREc/vsT53ToaICDl0uuwlNLlcQtmyz0NyUmB+5RUz3N2z+0wcV0bCtsEA7NqlwNq1SmzbpszwveT37snw7bdqfPutGq++akZwsBHduhlQtKi9ek+5hSGaiIiIiMhJubsDtWubUbv289BnMgH//ff8HuukWesnT6wvVdZoJLzyyotB2fK5fPm834faUalUQJs2JrRpY0J0tBbh4UqsXavCgQMZj1YXLsgxbZoc06erUbeuBs2aJaJPHwEaTQ52nOyGIZqIiIiIKB9RKIDAQBGBgSKCgy33WSft93zihBmXL8egUqVCqFZNjtKlJd6nmw1eXkDfvkb07WtEZKSAdetUCAtT4uLFjP1QJUnAoUMqHDpUFnPnSmjd2ojgYCOaNzeluega5S3HXp2AiIiIiIiyTRCAkiUltGxpQIcOj9CihQEBAQzQ9lS6tIRRo/Q4fDgee/fG4YMP9PDxyfje0TqdgA0bVOjZ0w2BgR4YN06DEyfkkBz3VvMCiyGaiIiIiIjITgQBqFFDxKxZOkRExGH9+gQEBxvg6prxNPz4sQw//KBG8+bueOMNd3zxhRrXrzO6OQqOBBERERERUQ5QKICmTU1YtkyLy5dj8f33iWjWzAiZLOOB+upVOWbP1qBmTQ+0bOmG5ctVePKES3vnJYZoIiIiIiKiHObuDnTvbsTvvyciIiIOn3+uRfXqaW+59bJjxxQYM8YFlSp5oFcvV/zxhwI6XQ51mFLFEE1ERERERJSLiheX8OGHBuzbF489ex7h3Xfvws8v44HaaBSwdasS/fq5oWJFT3z0kQsOHpRDzPgt2JQNDNFERERERER5JDDQjA8/vIPjxx9h8+Z49OljgKdnxi/3jo0VsHKlCm3buqN6dQ9Mn67GpUuMeTmJW1wRERERERHlMZkMaNDAjAYNtJg7V4vt2xUIC1Nh504FTKaM3QN9+7YMCxZosGCBBkqlBHd3CW5uePZZgrs7nn1+Xvby97bqJn2vYHoEwBBNRERERETkUDQaoGNHEzp2NOHxYwEbNigRFqbE8eMZj29Go4CnTwU8fWrPflkCdVLYth24Xwziz8te/N7TU4KPj/Pu3cUQTURERERE5KC8vSW8/74B779vwLVrMoSFKbF2rRLXr+f+Jt86nQCdTsDjx9lrp3p1M/bti7dPp/IAL5bPB8LDw9GxY0cEBATAy8sLN2/ezOsuERERERGRnZUrJ+KTT/Q4dSoeO3fG4/339ShSxPlWE3Nzc95ZaIAhOl9ITExEvXr1MHHixLzuChERERER5TBBAN5804x583S4eDEOv/ySgI4dDVCrnSOcurs7Rz9Tw8u584EePXoAACIiIvK4J0RERERElJtUKqBNGxPatDEhJkaLbduUiIiQIz4eiI8XEB8vICFBQEICkJAgIC7u+dcGQ8YWLLM3Z5+JdooQHRUVhY0bN2Lnzp24cuUK7t+/j8KFC+Ott97CiBEj8MYbb+R6n8LCwnD48GGcPn0aERERMBgM+PbbbxESEpLqY06dOoXZs2fj6NGjMJlMqFy5Mj788EN06tQpF3tORERERET5UaFCQI8eRgDGDNU3GCxhOilwJ4VtS9C29X3KYB4fj+R6SeHcbE4/mLu7Z/OJ5jGnCNHLli3DwoULUbZsWbz99tsoWrQorl69ii1btmDLli348ccf0blz51zt08yZMxEZGQlvb2/4+voiMjIyzfr79u1Dly5doNFo0LlzZ7i7uyM8PBz9+/fH7du3MXz48FzqORERERERkWUWW6WSULgwAGR/dliSAL0+acb7ecC2BO/n31eoYM72ufKSU4ToWrVqYfPmzWjQoEGK44cOHUKHDh0wevRotG3bFmq1Os12fv31V9SrVw/+/v42y81mM5YuXYpBgwZBpVKl2daiRYtQrlw5+Pv746uvvsK0adNSrWsymTBixAjIZDJs2bIF1atXBwCMHz8ezZo1w4wZM9ChQ4cU/Zo6dSoWLlyYZh+io6PTLCciIiIiIsotgmDZnkujkeDtDdgjmDsipwjR7du3t3m8Xr16aNiwIXbv3o2IiAjUrFkz1Tbu3LmDESNGwNfXF5s3b7YK0qIoYujQoVi7di1UKhUGDRqUZp+aNGmS4f7v27cP169fR0hISHKABoBChQph9OjR+OCDD7BmzRpMmDAhuWzYsGHo1atXhs9BREREREREOc8pQnRalEolAEAuT3ufNL//t3fvcVVV+f/HXyCaDuAlEUhAES8peSnNUhnvmo9EFNFUcrw9UpHQ0dEcp6bxa8UMWpY9xtLE1ETxRkVKOqbhBQKytBQaUhBREdMcFRBMAeH3Rz/OzInbOXpQLu/n48Hj4Vl7rbXXPtu19/nstfdeLi58+OGHTJs2DR8fHz7//HPc3NyAXwPoWbNmsWPHDiZMmMD06dMt2savvvoKgEGDBpVaNnjwYADi4uKM0h0cHHBwcLBoO0REREREROTe1OgprjIyMjh06BDOzs489thjleb38fFh3bp1ZGZmMmLECDIyMoxGoMeNG8eqVauwtrbs15KWlgZA27ZtSy1zcnLCzs6OM2fO3HX9169fJzExkdTUVABOnTpFYmIi169fv+s6RUREREREpLQaOxJdUFBAQEAAt2/fZsmSJZWORJcYNWoUd+7cYcaMGfj4+PDEE08QGRnJmDFjWL16tcUDaICcnBwAGjduXOZye3t7Q567sWfPHoKCggyfx40bB1Dp28JFRERERETEPDUyiC4qKuLFF18kPj6eKVOmGOZJNpWfnx+FhYXMnDmTs2fP4u3tTWhoqMmBeHUzceJEBcsiIiIiIiL3QY27nbuoqIigoCAiIiIYN24cK1asMLuO4uJiYmJiDJ9PnjzJzz//bMlmGikZgS5vtPnGjRvljlKLiIiIiIhI9VGjguiSEeitW7cyduzYu7r9uri4mLlz57J582b8/PwIDQ3l7Nmz+Pj48NNPP1VJu0uehS55Nvp/Xb58mdzcXDw8PKpk3SIiIiIiImI5NSaILgmgt23bhp+fH2vWrDH79uvi4mLmzZtHWFgYo0ePZu3atYwbN441a9aQnp6Oj48Ply5dsnjbvby8ADhw4ECpZdHR0UZ56rqaekt9baZ9Ur1of1Q/2ifVj/ZJ9aN9Ur1of1Q/2ic1S40Ioktu4d62bRu+vr539fxycXEx8+fPZ+PGjYYAuqSOsWPHGgXSly9ftmj7+/fvj7u7Ox9//DGJiYmG9OzsbN555x0aNGhg9nPdtVHDhg3x8PCgYcOGD7op8v9pn1Qv2h/Vj/ZJ9aN9Uv1on1Qv2h/Vj/ZJzWOVlZVV/KAbUZmQkBCWLVuGnZ0ds2bNKjOA9vb2pmvXruXWcfHiRfr374+XlxcffvghNjal36kWERHBrFmzWLVqFePHj6+wTWFhYSQkJACQnJzMiRMn6NWrF23atAGgd+/eTJ482ZA/JiaGMWPG0LBhQ/z8/LCzs2PXrl1kZGTwxhtvMGfOHJO+CxEREREREXlwakQQHRgYyNatWyvMY8p0TmfPnsXV1bXMALrE6dOnadeu3T23yd/fn9WrVxulHTt2jJCQEL755hsKCgrw9PQkKCgIPz+/StcnIiIiIiIiD16NCKJFREREREREqoMa8Uy0iIiIiIiISHWgIFpERERERETERAqiRUREREREREykIFqq1Hfffcdzzz1Hq1ataNmyJUOGDCEyMtKsOm7fvs2yZcvo3r07Tk5OdOzYkblz53LlypUqanXtc/HiRVatWsXo0aPp3LkzLVq0oEOHDkyaNImjR4+aXE9sbCxNmzYt9y88PLwKt6L26dKlS7nfpbe3t1l17dixg0GDBtGyZUtat27N+PHjOX78eNU0vJYKDw+v8P9306ZNGTlyZKX1qJ+Yb/v27cybN48BAwbg6OhY6feUk5PDK6+8QufOnXF0dKRLly787W9/Izc31+x1R0dHM3z4cFxdXXFzc2PEiBEcPnz4XjanVjB1nxQUFLBz505mzZrFU089hYuLC66urgwePJh169Zx584ds9ZryeNibWNOPwkJCanwOHTu3Dmz1n369GmmTp2Kh4cHzs7OeHl5sW7dOoqL6/arlczZJ5WdX5o2bcqFCxdMWq/6yYNX/muqRe5RedN6TZs2jQsXLpg0rVdRURHPP/880dHR9OzZk5EjR5KWlkZYWBiHDx/myy+/xMHB4T5sTc0WGhrKu+++S5s2bRg4cCAODg6kpaWxe/dudu/ezYcffmjWW+K9vLz4/e9/Xyq9S5culmx2ndC4cWMCAwNLpbdq1crkOpYvX05wcDBubm5MmzaN3NxcPv30U4YNG8bOnTvp1auXJZtca3Xp0oVFixaVuWzXrl38+OOPDB482OT61E9MFxwcTEZGBs2bN8fJyYmMjIxy8+bl5eHt7U1SUhKDBg1i7NixJCYmsnLlSuLi4tizZ4/Jc61u376dgIAAHBwc8Pf3ByAyMhJfX18++ugjRo0aZZHtq4lM3Sfp6elMmTIFOzs7+vXrx7PPPktOTg579+5lwYIF7Nu3j23btmFlZWXyui1xXKyNzOknJfz9/cv83po0aWLyek+ePMkzzzzDrVu38PX15ZFHHmHfvn0sWLCAkydP8tZbb5m1HbWJOfukvPNLeno6O3bsoGPHjri6upq8bvWTB0tv55YqUVhYSM+ePbl48SL79+83zOGdnZ3N4MGDOX/+PEePHq20o2/evJnZs2czduxY1q5dazgJr1+/nvnz5zN16lTefffdqt6cGm/Xrl08/PDDpX7Qx8fHM2rUKGxtbTl16hQPPfRQhfXExsbi4+PDokWLePnll6uyyXVCSTCVlJR013WkpaXx9NNP4+7uTnR0tOGHUWJiIkOHDsXd3Z2EhASsrXXj0d3Kz8+nY8eO5OTkkJycjKOjY4X51U/Md+jQITw8PGjVqhUrVqzgtddeK3fqyn/84x+8+eabzJs3jyVLlhjSlyxZwrvvvsvixYuZP39+pevMysqiW7du2NjYEBMTg4uLCwCZmZn069cPgOPHj2Nvb2+ZjaxhTN0nFy9eZM+ePfj7+2Nra2tIz8vLY8SIEXz//fd89NFH+Pr6mrReSxwXaytz+klISAjLli0jKiqKvn373tN6hw8fTnx8PBEREQwdOhT49bg4atQoEhIS2LdvH0899dQ9raOmMmeflGfhwoWsXbuW4OBgZs+ebVIZ9ZMHT7+qpErExMSQnp7O2LFjDQE0/Hrlc/78+eTn51c69zdAWFgYAIsXLza6ij1t2jTc3d2JiIjgl19+sfwG1DIjR44sc0SsT58+9O3bl6ysLJKTkx9Ay+RehYeHU1hYyIIFC4xGFrp27cqYMWM4deoUCQkJD7CFNd/u3bu5du0aw4YNqzSAlrszYMAAk0ZPiouL2bRpE3Z2dixcuNBo2cKFC7GzszOcNyrz2WefkZ2dzcyZMw0BNICLiwszZszg6tWrfP755+ZtSC1i6j5p2bIl06dPNwqgAWxtbQkKCgIgLi6uStpY15i6Tyzp9OnTxMfH07dvX0MADdCgQQP++te/ArBx48b72qbq5F73ya1bt4iIiKBBgwZMmDDBgi2TqqbbuaVKfPXVVwAMGjSo1LKS2yErO6neunWLo0eP0r59+1IHKCsrKwYOHMiGDRv4/vvv6dOnj4VaXvfUr18fgHr16plc5syZM6xatYpbt27RsmVL+vXrR8uWLauqibVafn4+4eHhXLp0CXt7e7p3786TTz5pcvnK+tqWLVuIi4vDy8vLYm2ua0qCssmTJ5tVTv3E8tLS0vjpp58YPHhwmUHb008/TXR0NBcuXKj0tsjK+s7SpUuJi4sz3OYt5rub8wvc+3FR/is+Pp5jx45hbW2Nh4cHAwYMwM7OzuTyFfWT3r17Y2trq4sk9yAqKoqsrCxGjRpl9uOJ6icPloJoqRJpaWkAtG3bttQyJycn7OzsOHPmTIV1pKenU1RUhIeHR5nLS9LT0tIURN+ljIwMDh06hLOzM4899pjJ5SIiIoiIiDB8trGxYebMmbzxxhtm/1iq6y5fvmwYrSnRvXt31q1bR5s2bSotn5aWhp2dHU5OTqWWlfS/kv4o5jt//jyHDx/GxcWFIUOGmFVW/cTySv4vV3ReiI6OJi0trdIguqLzlPqOZWzevBkoOwCryL0eF+W/QkJCjD43adKEpUuXmnxxqKI+V69ePVq3bs3JkycpLCzExkZhhbk2bdoEmH+RFtRPHjTdzi1VIicnB/j1pQdlsbe3N+SprI7yXn5RUndl9UjZCgoKCAgI4Pbt2yxZssSkH/UODg4sWbKEhIQEMjMzSU1NJTw8HA8PD1atWsXixYvvQ8trj4kTJ7Jz505SU1O5ePEiMTExjB8/nu+++46RI0dy48aNSuvIycmpsJ+V5JG7Ex4eTlFREf7+/iYHvuonVceS54WKzlPqO/fuo48+Yv/+/fTr149nnnnG5HKWOC4KdO7cmffee4/jx49z6dIlTpw4wZtvvomVlRUvvvgie/bsMameyvqcvb09RUVFd/Vm/Lru7NmzxMbG4urqysCBA80qq37y4OmSkUgdVFRUxIsvvkh8fDxTpkwx+TmcTp060alTJ8NnW1tbvL29efLJJ/Hy8mLNmjXMmzePFi1aVFXTa5W//OUvRp+7du3KmjVrgF/fGrxx40aTXzIilldUVER4eDhWVlb84Q9/MLmc+onUdXv37mXhwoW4ubkRGhpqVlkdFy3Dx8fH6HPr1q2ZOXMmjz76KL6+vgQHBzN8+PAH1DqBX+/UKC4uZuLEiWa//FP95MHTSLRUicpGA27cuFHu6Nlv68jOzi5zeWWj3VK2oqIigoKCiIiIYNy4caxYseKe63RycmL48OEUFhaaNe+0lG3atGkAHDlypNK8jRs3rrCfleQR8x06dIgLFy7Qr18/3N3d77k+9ZN7Z8nzQkXnKfWdu7dv3z6mTJmCo6MjUVFRODs7W6Rec46LUr7+/fvTpk0bkpOTTbrTorI+d+PGDaysrMx6zlp+/S22detWrK2tzbpIWxn1k/tHQbRUiYqeJ7t8+TK5ubnlPtNWwt3dHWtr63KfnS5JL+t5NilbyQj01q1bGTt2LKtXr7bY1EfNmzcH4ObNmxapry4z57ts27Ytubm5XL58udSyip75lMrd7QvFKqJ+cm9K/i9b4rxQ0XlKfefufPHFF0yaNInmzZsTFRVlkYtPJdR3LKfkuzRldpOK+tydO3c4d+4crVu31vPQZvryyy/JzMxk4MCBuLm5Waxe9ZP7R0G0VImSNwEfOHCg1LLo6GijPOVp1KgRPXr0IDU1lfPnzxstKy4u5uDBg9ja2vLEE09YqNW1W0kAvW3bNvz8/FizZo1FX25UMrJ2v6ffqI3M+S4t0dektGvXrrFnzx6aNWvGiBEjLFav+sm9adu2LY888ghHjhwhLy/PaFleXh5HjhyhdevWlb5UDNR3LO2LL75g8uTJNGvWjKioqEovlJtLfccy8vLyOHnyJLa2toaAqyIV9ZOEhATy8vLUT+7CvbxQrCLqJ/ePgmipEv3798fd3Z2PP/6YxMREQ3p2djbvvPNOqfnwLl26REpKSqnbhaZMmQLA66+/TnFxsSF9w4YNnD17lueee45GjRpV8dbUfCW3cG/btg1fX19CQ0MrDKCvXr1KSkoKV69eNUo/fvx4mflXr15NbGwsbdu2pXv37pZseq2VkpJS5pXilJQUlixZAsDYsWMN6dnZ2aSkpHDp0iWj/BMnTsTGxoa3337bqP8kJibyySef8Oijj9K7d++q2YhabNu2beTn5zNu3DgeeuihMvOon9x/VlZWTJo0idzcXN566y2jZW+99Ra5ubmG80aJmzdvkpKSQkZGhlH66NGjady4MaGhoWRmZhrSMzMzWbt2Lc2bN7foBZTabP/+/UyePJmmTZsSFRVV6Qh+QUEBKSkppKenG6Wbe1yUst24cYPTp0+XSv/ll1+YO3cuN27cwNfXt9TocUpKCikpKUZp7du3p0+fPsTGxrJ//35Den5+Pn//+98ByweCtd1//vMf9u7di4ODA88++2y5+dRPqjerrKys4sqziZgvJiaGMWPG0LBhQ/z8/LCzs2PXrl1kZGTwxhtvMGfOHEPewMBAtm7dyvvvv8/EiRMN6UVFRTz33HNER0fTs2dPvLy8OHPmDFFRUbRq1Yro6Giz59Wri0JCQli2bBl2dnbMmjWrzADa29ubrl27GuVftGgRL7/8siFPly5dqF+/Pk888QQtW7bk5s2bfPvttyQmJtKkSRM+/fRTevTocd+2qyYLCQlh1apV9OnTBzc3N373u99x+vRp9u/fT0FBAfPnzzd6i3N4eDhBQUH4+/uzevVqo7qWL19OcHAwbm5ujBw5ktzcXD799FPy8/PZuXMnvXr1ut+bV+P16dOH5ORk4uLiyp3+Tf3EcsLCwkhISAAgOTmZEydO0KtXL8M0Lb179zb8UM/Ly2PYsGH88MMPDBo0iG7dunHixAkOHDhA9+7d2b17t9HF1djYWHx8fPDy8mL37t1G692+fTsBAQE4ODgwevRoACIjI7l69SobNmzA19f3Pmx99WTqPklJSaFv377cvn2bMWPG0K5du1J1tWrVyujcfu7cObp164abmxtJSUmGdHOPi3WNqfvk3LlzPP7443Tv3p0OHTrg5OTEzz//zOHDh8nMzMTT05PPP/+chx9+2Kj+pk2bApCVlWWU/uOPPzJs2DBu3brF6NGjcXZ2Zt++ffz444/MmDGj1AWtusScY1eJlStX8re//Y2goCDDhYiyqJ9Ub3qAQapMv3792Lt3LyEhIURGRlJQUICnpyevvfYafn5+JtVhbW3Nli1bWLFiBdu3b2fVqlU0a9aMSZMm8eqrryqANlHJ7fC5ubksX768zDytWrUyBNHleeGFF4iOjiY+Pp5r165hbW2Nm5sbgYGBzJ49GxcXF4u3vbbq27cvKSkpJCYmkpCQwM2bN2nevDlDhw5l+vTpZs2r+tJLL9GqVStWr17N+vXrqV+/Pr179+aVV17h8ccfr7qNqKWOHTtGcnIyPXr0MGv+9BLqJ+ZLSEhg69atRmlff/01X3/9teFzyQ9RW1tbdu/ezdKlS4mKiiI2NhYnJydmz57NokWLzLo7afz48TRv3py3336bLVu2YGVlRbdu3Vi4cCEDBgywyLbVVKbuk8uXL3P79m0APvnkkzLr8vLyMgqiy2PJ42JtZOo+adasGdOnT+fYsWPs37+frKwsGjVqRIcOHQgICGDGjBlm9ZNOnToRHR1NcHAw+/bt4+bNm7Rt25bly5fzwgsvWGz7aiJzjl0lSuZPv9sRfPWT6kEj0SIiIiIiIiIm0jPRIiIiIiIiIiZSEC0iIiIiIiJiIgXRIiIiIiIiIiZSEC0iIiIiIiJiIgXRIiIiIiIiIiZSEC0iIiIiIiJiIgXRIiIiIiIiIiZSEC0iIiIiIiJiIgXRIiIiIiIiIiZSEC0iIiI11rlz52jatClNmzZ90E0REZE6wuZBN0BERESqlre3N3FxcSblzcrKqtrGiIiI1HAKokVEROoIV1dXXF1dH3QzREREajQF0SIiInXExIkTefnllx90M0RERGo0PRMtIiIiIiIiYiIF0SIiIlLKb1/Y9a9//Qtvb29at26Ni4sLQ4YMYceOHRXWcerUKYKCgujSpQuOjo60bt2a4cOHExYWxp07d8otV1hYyJYtW/Dz86Ndu3Y4OjrSqVMnhg8fzsqVK8nOzi63bEJCAuPGjaNNmzY4OzvTp08fQkNDKS4uLjP/8ePHmTFjBp07d8bR0REXFxe6dOnCmDFjWLlyZbnlRESk7tLt3CIiIlKhNWvWsGjRIpo1a4aHhweZmZkcPXrU8Pfmm2+WKhMZGUlAQAD5+fnY2tri6enJ9evXiY+PJz4+nsjISLZs2UKjRo2Myl29epXnn3+eI0eOAODk5ETnzp25cuUKR44cIT4+nscff5y+ffuWWmd4eDhz5syhSZMmuLu7k5GRQXJyMn/+8585f/48wcHBRvm//PJL/P39KSgowM7Ojnbt2mFjY8PFixeJjo4mOjqawMBAbGz0c0lERP5LI9EiIiJSoVdffZWFCxeSmprKwYMHOXXqFO+88w7W1taEhoby2WefGeVPSUkhMDCQ/Px8Jk+eTEpKCocOHeLEiRN89tlnNG7cmIMHD7J48WKjcsXFxUyZMoUjR47g6urKzp07OXXqFAcOHCApKYkzZ87w9ttv4+joWGY758+fT3BwMKdPn+bgwYOcPn3asI7333+f9PR0o/xLliyhoKCAuXPnkpqaSnx8PDExMZw+fZqkpCRee+01rK31U0lERIxZZWVl6T4lERGRWszUKa6GDx/Oli1bgF9v5+7WrRsAAwcOJDIyslT+uXPnsnHjRjw9PYmPjzekz549m82bN+Pp6UlcXBxWVlZG5cLCwvjjH/9I/fr1SUpKwtnZGfj1lnF/f38eeughYmJiePTRRytt8/+2c9KkSaxcubJUnj59+pCcnMzSpUuZNWuWId3JyYnbt29z7tw5mjRpUum6REREQCPRIiIidYarqyu9evUq969jx45llgsMDKwwPTk5mQsXLhjS9+/fD8CsWbNKBdAA/v7+tGjRgoKCAg4ePGhI37VrFwA+Pj4mBdC/NX369DLTn3rqKQDOnDljlO7m5gbAxx9/bPa6RESk7tJDPiIiInXE3U5x1alTpzLT27dvj42NDYWFhaSkpODq6kp2djaXL18GwNPTs8xy9evXp3379ly5coXU1FRDenJyMvDfoNdc7dq1KzO9RYsWAOTm5hqlz507lzlz5rBgwQLee+89Bg4cSM+ePfHy8qJVq1Z31QYREan9NBItIiIiFSrvGeR69erx8MMPA3Djxg3AOFAtrxxguIW7pNz//vtub622tbUtM73kuebfvml70qRJbNq0iaeffppz586xfv16AgMD6dq1K0OGDCE2Nvau2iEiIrWbgmgRERGp0M8//1xm+p07d7h27RoA9vb2ANjZ2VVaDuDSpUtG5f733xVNYWVpPj4+fPHFF6SnpxMREcGf/vQn2rRpw9GjRxkzZgxJSUn3rS0iIlIzKIgWERGRCp08ebLM9NTUVAoLCwHo0KED8OsospOTE/Df27N/q7Cw0HAbd0k5gMceewyAb775xjINN0OTJk0YOnQo//d//8e3335Lz549yc/PJyws7L63RUREqjcF0SIiIlKhDz74oMJ0T09PXF1dDenPPPOMYflvb6EG2LZtG1euXKF+/foMHDjQkD5q1CgAoqKijJ6Vvt9sbGzo0aMHAD/99NMDa4eIiFRPCqJFRESkQjExMSxbtsww6lxcXMzGjRvZtGkTAC+99JJR/jlz5tCwYUOSk5OZN28eeXl5hmWHDx/m1VdfBWDq1KmGUWuAYcOG0a9fP27fvo2fn1+pZ5JzcnJYv349p06duudtysnJYcqUKURHR5Ofn2+07Pjx44Ypvbp3737P6xIRkdpFb+cWERGpI8LDwzl8+HCFeZYtW2aYd7lEcHAwixYt4oMPPqBNmzZkZmYa3sA9ffp0/Pz8jPJ36NCB1atXExAQwMaNG/nkk09o3749169f5+zZs8Cvc0+//vrrpda/YcMGJkyYwLfffouPjw/Ozs64uLhw5coVMjMzuXPnDlFRUXc1Bdb/KioqYufOnezcuZMGDRrg4eGBra0tV65c4fz58wA8+eSTRvNKi4iIgIJoERGROuPChQtG8zmXJScnp1RaQEAAbm5uvP/++yQlJVFYWEiPHj2YMWMGEyZMKLOe0aNH06lTJ/75z38SExPDv//9bxo2bEjv3r3x9/dn4sSJ1KtXr1S55s2bs2fPHrZu3UpERAQ//PADSUlJODg40KtXL7y9vUsF+XfD3t6etWvXEhMTw7Fjx7h06RLZ2dnY29vTu3dvRo8ezdSpU2nQoME9r0tERGoXq6ysrNIPK4mIiEiddu7cOUOwmpWV9WAbIyIiUo3omWgREREREREREymIFhERERERETGRgmgREREREREREymIFhERERERETGRXiwmIiIiIiIiYiKNRIuIiIiIiIiYSEG0iIiIiIiIiIkURIuIiIiIiIiYSEG0iIiIiIiIiIkURIuIiIiIiIiYSEG0iIiIiIiIiIkURIuIiIiIiIiYSEG0iIiIiIiIiIkURIuIiIiIiIiY6P8BniIzS9V6aVYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_emb.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZCKFSECRPIpW",
    "outputId": "b477d2d6-f513-4c01-a04d-4633fbc33ef8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[423, 447],\n",
       "        [271, 324]])"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_emb.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_kqa22QNPIpX"
   },
   "source": [
    "## Model II - GloVe + Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "id": "MH3DdVEFPIpX"
   },
   "outputs": [],
   "source": [
    "class TransfClassifier(nn.Module):\n",
    "    def __init__(self, embedding_layer, encoder, n_outputs):\n",
    "        super().__init__()\n",
    "        self.d_model = encoder.d_model\n",
    "        self.n_outputs = n_outputs\n",
    "        self.encoder = encoder\n",
    "        self.mlp = nn.Linear(self.d_model, n_outputs)\n",
    "\n",
    "        self.embed = embedding_layer\n",
    "        self.cls_token = nn.Parameter(torch.zeros(1, 1, self.d_model))\n",
    "\n",
    "    def preprocess(self, X):\n",
    "        # N, L -> N, L, D\n",
    "        src = self.embed(X)\n",
    "        # Special classifier token\n",
    "        # 1, 1, D -> N, 1, D\n",
    "        cls_tokens = self.cls_token.expand(X.size(0), -1, -1)\n",
    "        # Concatenates CLS tokens -> N, 1 + L, D\n",
    "        src = torch.cat((cls_tokens, src), dim=1)\n",
    "        return src\n",
    "\n",
    "    def encode(self, source, source_mask=None):\n",
    "        # Encoder generates \"hidden states\"\n",
    "        states = self.encoder(source, source_mask)\n",
    "        # Gets state from first token only: [CLS]\n",
    "        cls_state = states[:, 0]  # N, 1, D\n",
    "        return cls_state\n",
    "\n",
    "    @staticmethod\n",
    "    def source_mask(X):\n",
    "        cls_mask = torch.ones(X.size(0), 1).type_as(X)\n",
    "        pad_mask = torch.cat((cls_mask, X > 0), dim=1).bool()\n",
    "        return pad_mask.unsqueeze(1)\n",
    "\n",
    "    def forward(self, X):\n",
    "        src = self.preprocess(X)\n",
    "        # Featurizer\n",
    "        cls_state = self.encode(src, self.source_mask(X))\n",
    "        # Classifier\n",
    "        out = self.mlp(cls_state) # N, 1, outputs\n",
    "        return out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "id": "aUum393hPIpX"
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(33)\n",
    "# Loads the pretrained GloVe embeddings into an embedding layer\n",
    "torch_embeddings = nn.Embedding.from_pretrained(extended_embeddings)\n",
    "# Creates a Transformer Encoder\n",
    "layer = EncoderLayer(n_heads=2, d_model=torch_embeddings.embedding_dim, ff_units=128)\n",
    "encoder = EncoderTransf(layer, n_layers=1)\n",
    "# Uses both layers above to build our model\n",
    "model = TransfClassifier(torch_embeddings, encoder, n_outputs=1)\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "id": "2mBjsLwHPIpX"
   },
   "outputs": [],
   "source": [
    "sbs_transf = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_transf.set_loaders(train_loader, test_loader)\n",
    "sbs_transf.train(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "4CwpEol1PIpY",
    "outputId": "2e6f1481-7f35-4fd2-d26b-2921f7a28c25"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACQG0lEQVR4nOzdd3gUVdsG8Ht2N1vSgUBoCaFD6CpKBwH1kxCBhCqCIgpEQIoIiijwgiBKlRKkSZUiitIVRQhVEKSGJi2BQKjp2WyZ+f4ICdnspm+2kPt3XbmAOVOe5B3nzb3nzDlCXFycBCIiIiIiIiLKk8zeBRARERERERE5C4ZoIiIiIiIionxiiCYiIiIiIiLKJ4ZoIiIiIiIionxiiCYiIiIiIiLKJ4ZoIiIiIiIionxiiCYiIiIiIiLKJ4ZoIiIiIiIionxiiCYiIiIiIiLKJ4ZoIiIiIiIionxiiCYiIiIiIiLKJ4ZochharRbXrl2DVqu1dylEOeJ9Ss6A9yk5A96n5Cx4r1J2DNHkUIxGo71LIMoT71NyBrxPyRnwPiVnwXuVsmKIJiIiIiIiIsonhmgiIiIiIiKifGKIJiIiIiIiIsonhmgiIiIiIiKifGKIJiIiIiIiIsonhmgiIiIiIiKifFLYuwAiIiIiInpKFEUkJydzXWIHIYoilEol4uPjkZiYaO9yqIDUajXc3Nwgk1mv/5ghmhxKeHhFaDTuaNcOaNbMAA8Pe1dEREREZDuiKOLhw4dwd3eHj48PBEGwd0klniiK0Ol0UCqVVg1iVPwkSYJWq8XDhw9RpkwZq/3vxxBNDkOSgJ9/Lou4OBcsXAjI5RIaNzaiVSsjWrUyMFQTERHRMy85ORnu7u7QaDT2LoXI6QmCkPnfUnJyMjysFCYYoslhXLokR1ycS+a/jUYBJ04ocOKEAvPmqRiqiYiI6Jmn1Wrh4+Nj7zKInilqtRoPHjxgiKZnz+HDylzbGaqJiIioJOAQbiLrsvZ/UwzR5DDyCtHZMVQTEREREZGtMUSTw3B3l+DubkBSUuFuy7xC9UsvGeDpaeWiiYiIiIioROH0cuQw5s5NwB9/nMLvvz/E1Kmp+L//08PTUyr0+TJC9bx5KvTo4YaqVT3RoYMbJk5UY88eBRISrFg8ERERETklb29vBAUFFekcBw4cgLe3N6ZPn26lqsiRsSeaHIpcDjRsaMCLL+owbJgORiNw9qwMBw8qcPCgAocPK5CQULh3GthTTUREROSYvL29C7R/XFxcsdTxLGnQoAHu3buH2NhYe5fyzGGIJocmlwONG4to3JihmoiIiOhZNW7cOLNt4eHhSEhIsNhmTceOHSvykmLPP/88jh07hjJlylipKnJkQlxcXOHHyxJZkVarxf2ICFT9808Yx4yBVLZsnscYjcC5c09D9aFDhQ/V2TFUkyVarRbR0dHw8/ODWq22dzlEFvE+JWfA+9Sy+/fvo2w+fgcqCRo0aIDo6Gi79zqLogidTgelUgmZzHnehmVPtClr/rflPHcBlQgVlyyB23ffwaNxY6imTAHyeGjK5UCjRiKGDtVh/foUXL+egP37E/Hll6l4/XXrvlMdEOCJ9u3d8MUXavz+O9+pJiIiIrK1mzdvwtvbG2FhYbh06RL69u2LqlWrwtvbGzdv3gQAbNu2DQMHDkSTJk1QoUIF+Pv74/XXX8evv/5q8ZyW3okOCwuDt7c3bty4ge+++w6tWrVC+fLlUb9+fXz11VcQRdFk/5zeiW7QoAEaNGiApKQkjBs3DnXq1EG5cuXQokWLHOu5efMmBgwYgICAAFSqVAmdOnXCoUOHMH36dHh7e+PAgQOF/fHlKDk5GdOmTUPTpk3h6+uLgIAA9OzZE0ePHjXbV6vVYv78+WjZsiX8/f1RsWJFNGjQAO+88w7Onj2buZ8oili9ejXat2+PgIAAlC9fHoGBgejVq1exfA+2xOHc5DAUZ8+i9B9/AACE5GSoZ82CaulSpA0fjrQhQ5Cf9aoyQnWjRjoMHaqzak+1KAo4eVKBkycV+PZbFWQy8yW12FNNREREVPyuX7+OV155BYGBgXjzzTfx6NEjKJXpy6X+73//g4uLC5o1a4by5cvjwYMH2LVrF95++23MmDEDgwcPzvd1vvjiCxw6dAgdO3ZE+/btsXPnTnz11VfQ6/X4/PPP83UOg8GAkJAQxMXFITg4GKmpqfj555/xzjvv4KeffkL79u0z942JicFrr72Gu3fvomPHjmjYsCGuXLmCbt26oU2bNgX7IeWTVqvFG2+8gRMnTqBRo0YICwvDvXv3sGXLFvz5559Yvnw5unbtmrl/WFgYtmzZgnr16uHNN9+ESqXC7du3ceDAAfz7779o0KABAGDy5MmYN28eqlatih49esDd3R0xMTE4evQo9u/fj9atWxfL92MLDNHkMNy//tpsm5CQAPWXX0K5eDHSRo6E7r33gAK8s5JXqD58WIH4eIZqIiIicg6vvOJm7xJytGdPss2udfToUYwdOxbjx483a/vxxx8REBBgsi0pKQmvvvoqvvzyS/Tr1w+urq75us7p06dx4MABlC5dGkqlEuPGjcNzzz2HJUuWYNy4cZnBPTd37txBkyZNsH379sz9e/TogS5dumDhwoUmIXrSpEm4e/cuPv/8c3z00UeZ29esWYPhw4fnq+aCmjdvHk6cOIGePXviu+++gyCk/248ePBgvPLKKxgxYgQ6dOgADw8PxMfH45dffkHjxo3x559/Qi6XZ57HaDQiMTEx89+rV69GhQoVcOjQIbOf9+PHj4vle7EVhmhyCPLjx6HasyfHdtnDh9B8/jlUCxcibcwY6Pr3B/Lx0DK7DkM1ERERObHjx/nrOwD4+vpizJgxFtuyB2gAcHd3x5tvvokJEybg5MmTaNWqVb6u8/HHH6N8+fLQ6XQAgDJlyqBTp05Yv349rly5gnr16uXrPNOmTTMJ3G3btoWfnx9OnjyZuS0tLQ2//vorypYti2HDhpkc/9Zbb+Hbb7/FlStX8nW9gli/fj1cXFwwceLEzAANAI0aNUKfPn2watUq7NixA71794YgCJAkCWq12uz9cLlcbjbLuouLi0nQzlCqVCmrfx+2xHeiySFIZcpAGxyc536yu3ehGTMGHs8/D5e1awGDoUjXzf5O9bVrpu9Ue3kV/p3qjFD97bcq9OzJd6qJiIiIrKV+/fo59gLfv38f48ePx4svvogKFSrA29sb3t7emDBhAgDg7t27+b5O48aNzbZVqlQJABAfH5+vc3h5eVkM9pUqVTI5x5UrV5CWloYmTZpApVKZ7CsIAl588cV8151fCQkJuHHjBqpVq5b5fWWVMeQ6411nT09PvPrqqzh69CjatGmDWbNm4e+//4Zerzc7NjQ0FFFRUWjevDmmTp2K/fv3IzU11erfgz3woyxyCGK1aohfuhRX//gDNVauhOrJu9E5kUVHw3XYMBjnzEHap59CHxICWGG2RPZUExERETm+nGZZfvz4MV5++WXcunULzZo1Q9u2beHl5QW5XI6zZ89i586dSEtLy/d1PCzMyZPRs2o0GvN1Ds8cfsGTy+UmE5RlDIX28fGxuH+5cuXydb2CyLhmTj9PX19fk/0AYOXKlZg9ezZ+/PFHTJkyBUD69/jmm2/iiy++yBy6/dVXX6FKlSpYt24dZs6ciZkzZ0KtVqNr16748ssvnXo5MIZociiptWsjbu1auJ05A/WXX0Kxf3+u+8uvXoXre+/BOHs2tOPHwxAUBAjWWeIKYKgmIiIickRCDr/vrVmzBrdu3cJnn32Gjz/+2KRtzpw52Llzpy3KK5SMwP7gwQOL7ffu3Su2a96/fz/Xa2b9MMHV1RUTJkzAhAkTcOPGDRw4cADff/89Fi9eDK1Wi7lz5wIAFAoFhg8fjuHDh+POnTs4dOgQ1q1bhw0bNuDevXv4+eefrf792ApDNDkk44svIvnXXyGPiIB66lQojh3LdX95ZCTc3noLhiZNkDZhAgzt21s1TGdeh6GaiIiI7Khp06K9yvasu379OgCgU6dOZm1HjhyxdTkFUrNmTahUKpw6dQppaWkmQ7olScLx48etfk1PT08EBATg2rVriImJQcWKFU3aDx48CACZM25nFxAQgICAAHTv3h01a9bErl27MkN0VhUqVED37t0REhKCF154Afv27UNqaio0BZgw2JEwRJNDM7Zpg+TffoNizx6op06F/MyZXPdX/PsvFKGhMDRvDu2ECTC2bFms9TFUExERkS3ZcgZsZ+Tn5wcgffburJN+/fjjj/j999/tVVa+qFQqdOnSBZs2bUJ4eDhGjhyZ2bZ+/Xpcvny5WK7bp08fTJ8+HZMnT8bixYsze/nPnTuHH374AZ6enpnraD948AD37t1DYGCgyTni4uKQlpaG0qVLA0ifJO3UqVN46aWXTPZLTk5GcnIyXFxczCYmcyYM0eT4BAGGV19F0iuvQLFtG9TTpkF+8WKuhyiOHIF7UBD0L7+MtAkTYHz+eZuUylBNREREZD+9evXC3LlzMXbsWBw4cAB+fn44d+4c9u/fj+DgYGzbts3eJebqiy++wL59+zBp0iQcOnQoc53o3377DR07dsQff/xRoPCp1+sRFhaWY3t4eDhGjBiB33//HRs3bsTly5fRtm1b3L9/H1u2bIHBYMB3332XOZw7JiYGbdq0Qf369VGvXj1UrFgRjx49ws6dO6HX6zOX4UpNTcVrr72GGjVqoHHjxqhcuTKSk5Oxe/duxMbGYvjw4WaTpzkThuhnwNatW7FixQqcOnUKcXFxOH36NKpUqWLvsqxPEGB44w0kBQXB5aefoJo+HfInQ3Zy4vLXX3D56y/oX38d2s8+g1i/vo2KTcdQTURERGQ7lSpVwo4dOzBx4kTs27cPRqMRDRs2xJYtW3Dr1i2HD9GVK1fG77//jkmTJmHv3r04dOgQGjVqhJ9//hm//PILAMuTneVEFEWsX78+x/bw8HCo1Wps3boVc+fOxZYtW7Bo0SJoNBq0bNkSo0ePRvPmzTP39/f3xyeffIKIiAjs378fjx49QpkyZdCoUSMMGTIEHTt2BAC4ublh8uTJ2L9/P44cOYL79+/D29sbNWrUwMSJExEaGlq4H5CDEOLi4gq/hg85hA0bNiAqKgre3t4YO3as04ZorVaL6Oho+Pn5Qa1W532AXg+X9euh/vpryG7dytc1dCEhSPv0U4g1axaxWuuwZqjOTiaT0KiRES+/bMDAgTpUqsT/1K2hwPcpkR3wPiVnwPvUsvv37+c4UzLZhyiK0Ol0UCqVdh2C/H//9384duwYoqKi4O7ubrc6nJU1/9tiT/QzoHfv3gCAyMhIO1diYy4u0PfvD33PnlCuWgXVrFmQ5TFrofLnn+Hyyy/Q9+4N7dixkCys2WdLxd1T/e+/Cvz7rwKrVyuxc2cyatUS8z6QiIiIiOzm7t27KF++vMm2jRs34ujRo2jfvj0DtANwure5t23bhq5du6Jq1arw9fVFw4YNMXDgQNzKZ0+ktWzcuBEjR45Eu3btUK5cOXh7e2PdunW5HnPy5En06NED/v7+qFixIjp27IgtW7bYqOJnmFoN3eDBSPz3X6ROngyxVKlcdxdEEcoffoBH06ZQf/QRhJgYGxWat4xQPXSoDuvXp+DatQRERCRi2rRUdOqkh5dX4XqTHzyQoUcPN9y7Z/0Zy4mIiIjIepo3b44uXbpg7Nix+PTTT9G5c2cMHjwYHh4emesyk305TU+0JEkYNWoUVq5ciapVqyI0NBTu7u6Za45FR0ejcuXKNqtn6tSpiI6ORpkyZeDr64vo6Ohc94+IiEBoaCjUajVCQkLg7u6OrVu3YsCAAbh161bmS/hUBG5u0I0YAd0770AVHg7VwoUQsiwMn52g10O1fDmU69ZBN3Ag0kaOhORgw6fkcqBhQxENG+rwwQfpPdXnzz/tqT50KP891TdvytCnjyu2bUuGq2sxF05EREREhfLuu+9i165d+Pfff5GSkgIfHx/06NEDH3/8MWrVqmXv8ghO9E50eHg4Pv30U7z33nuYMWMG5HK5SbvBYIBCkftnAhs2bECLFi3g7+9vsd1oNCI8PByDBg2CUqnM9Vz79u1DtWrV4O/vjzlz5mDy5MlYuHAh+vbta7avwWBA06ZNERMTgz179qBhw4YAgPj4eHTo0AFRUVH4559/TOqaNGmSxTXWsoqLizP5d2RkJFq0aFFy3onOg/DoEZTffgvVd99BSE3Nc3/JzQ1pQ4YgbfhwwNu7yNe3hcKE6qAgPVavTkG2/4Qon/gOHzkD3qfkDHifWsZ3oh2Po7wTTUVjzf+2nOIuSE1NxYwZMxAQEICvvvrKLEADyDNA3759GyNGjEDnzp0RFRVl1i6KIsLCwjBhwgSsXLkyz5ratWuXYxjPLiIiAtevX0f37t0zAzQAeHl5YfTo0dDpdGaz5g0bNgzHjh3L9YtyJ5UujbRJk5B46hTSBg+GlMcHI0JyMtSzZsGzYUOovvkGyKUX21Fk9FR/8IEOP/xgOvw7MNBo8ZgdO1wwYQJ/WSEiIiIiKgynCNF79+5FXFwcgoKCYDQasXXrVsyZMwcrVqzAtWvX8nWOSpUqYdmyZYiJiUFwcLDJ8GtRFDFkyBBs2rQJvXv3xnvvvWfV+g8ePAgAaN++vVlbhw4dAACHDh0y2e7j44NatWrl+kX5I/n6QjtjBhJPnoTu7bch5dEFKyQkQP3ll/Bo3BjK+fOBfPRiO4qsoXrr1mRUrWo5SIeHq7B4ce4fKhARERERkTmnCNGnTp0CAMjlcrRs2RL9+/fH5MmTMXr0aLzwwguYMGFCvs4THByM5cuX4/bt2+jcuTOio6Mze6A3bdqEnj17YtGiRVYfpnH16lUAQPXq1c3afH194e7unu8PAyx5/Pgxzpw5gytXrgAALl26hDNnzuDx48eFPuezSKpcGanz5iHp+HHoevaEJOQ+7Fn28CE0n38OjyZNoFy2DNDpbFSpdfj4SPjxxxSUKmV5Ru5PP1Vjxw6nmRaBiIiIiMghOEWIfvDgAQBg4cKF8PT0xN69e3Hr1i3s3LkTNWrUwIIFC7B8+fJ8natLly5YsmQJbt26heDgYAwcOBAbN25EaGgowsPDi+U9h4SEBACAp6enxXYPD4/MfQpj586daNOmDd5++20AQM+ePdGmTRvs3Lmz0Od8lonVqiF1yRIkHTkCfZcuee4vu3sXmjFj4PH883BZuxYwGGxQpXXUqCFi/foUqFTmUx9IkoD33nPFiRN8OZqIiIiIKL+cIkSLYnpPmlKpxLp16/Dcc8/B3d0dLVq0wMqVKyGTybBgwYJ8ny8kJATh4eG4ceMGtmzZgqCgICxZssTiu9bOoG/fvoiLizP7sjTJGT0l1qmDlFWrkLhvH/SvvZbn/rLoaLgOGwb3Zs3g8tNPgOgcay43a2ZEeLjlIempqQJ693bFjRtc+oqIiIiIKD+cIkRn9OA2btwYFSpUMGkLDAxEQEAArl+/bjZbdU4kSUJERETmvy9evIh79+5Zrd7sMurPqbc5MTExx15qKn5i48ZI2bgRSb/9BkObNnnuL//vP7gOHAj3Vq2g2LEDkBx/gvuQED0mT7YcpO/fl6FnTzfk8z8fIiIiIqISzSlCdM2aNQGkz2ZtScZ2rVab57kkScKIESOwdu1ahISEYMmSJbhx4waCg4Nx584d6xWdRca70BnvRmcVGxuLpKQkVKtWrViuTflnfOklJG/diqRff4XhxRfz3F8eGQm3vn3h1r49FH/+6fBh+sMPdRgwIM1i2+XLcvTt64Y0y81ERERERPSEU4To1q1bAwAuX75s1qbX63Ht2jW4ubnBx8cn1/NIkoSRI0di9erV6NatG5YuXYqePXviu+++w/Xr1xEcHIy7d+9avf6WLVsCSJ9lPLs///zTZB+yP2Pbtkj+7Tckb9oEY5YlyXKi+PdfuIWGwq1TJ8izzbLuSAQB+OYbLV55RW+x/dAhBYYP1zj6ZwFERERERHblFCG6atWqaN++Pa5du4bVq1ebtM2ZMwfx8fEICgrKda1oSZIwevRorFq1KjNAZ7wD3b17d5MgHRsba9X627Zti4CAAGzevBlnzpzJ3B4fH4/Zs2dDqVSid+/eVr0mFZEgwPDqq0jatw/Jq1bBWLt2nocojhyBe1AQXENCID9xwgZFFpxCAaxYkYKGDS0vfbVpkxJffqmycVVERERERM7DKUI0AMyaNQtly5bFhx9+iF69emHChAl44403MG3aNPj5+WHKlCm5Hn/nzh1s374dXbt2xdKlS80Cd/fu3bF48WJcu3YN+/bty7Oe1atXIywsDGFhYfjll18AAGvWrMncljXsKxQKfPvttxBFEUFBQRgxYgQ+++wztGrVCv/99x8+//xzVKlSpcA/E7IBmQyGLl2QdPgwUr77DsaAgDwPcdm7F+4dOsC1Tx/Izp0r/hoLyMMD2LgxGZUrW54YbeZMNVavdrFxVURERETFa926dfD29sa6detMtjdo0AANGjQo8nmsafr06fD29saBAweK7RpUeE4ToqtWrYq//voLb775Jk6dOoXvvvsO165dw/vvv4+9e/fC19c31+MrVqyIPXv2YNmyZTn2WPfo0QN///03evXqlWc9R44cwfr167F+/XqcPn0aAHD06NHMbUeOHDHZv02bNti9ezdeeuklbNmyBStWrEC5cuWwYsUKDB8+PJ8/BbIbuRz6Xr2QdPw4UubNg1ipUp6HuOzaBffWraEZOBCyJ2t4O4oKFSRs3JgMT0/LY7dHjdJg716uIU1ERES28d5778Hb2xubN2/Odb+EhARUqFAB/v7+SE21PGmqMzhw4AC8vb0xffp0e5eSLxmh/qeffrJ3KQ7BqX5Lrly5MhYtWlTo4wPy0YtYo0aNfJ0rPDwc4eHhBbr+888/n+eDgRyciwv0b78Nfa9eUK5cCdXs2ZDlMrO7IElQ/vQTXLZsgb5PH2jHjoXkIKMO6tUTsXp1Mrp3d4PBYLrEldEo4O23XbFrVxLq13eOpbyIiIjIefXr1w+bN2/G2rVr0b179xz3++mnn5Camoo+ffpAo9FY5dpbt261ynmsadCgQQgNDUXlypXtXQpZ4DQ90UQORa2GbsgQJP77L1InTYLo7Z3r7oIoQrluHTxeeAHqjz6CUEwzwRdUu3ZGzJ1r+VPcxEQBPXu6ISaGa0gTERFR8WrTpg2qVKmCiIgIREdH57jf2rVrAaSHbmupWrUqqlatarXzWUOZMmVQq1YtuLq62rsUsoAhmqgo3NygGzkSiadPQ/vJJ5A8PHLdXdDroVq+HB5NmkD92WcQHjywUaE5e+stPT7+2PLycDEx6WtIJybauCgiIiIqUQRBQN++fSGKYo7vGl+4cAEnTpxAvXr10KRJE8THx2Pu3Lno1KkT6tSpg7Jly6JOnToYPHgwrl+/nu9r5/RO9OPHjzFq1CjUrl0bVatWRYcOHbBt27Ycz7NmzRr06dMHDRo0gK+vLwICAhASEoKIiAiT/aZPn47g4GAAwIwZM+Dt7Z35dfPmzcx9cnoneteuXejcuTP8/f1Rvnx5tGzZEgsWLIDBYDDZ7+bNm/D29kZYWBiuXbuGvn37okqVKqhYsSK6dOmCs2fP5vtnVFD5rREAIiIi0L17d9SpUwflypVDzZo18frrr2PlypUm+506dQr9+/dH/fr1Ua5cOVSvXh0vv/wyZs6cWWzfR06cajg3kcPy8kLaJ59AN2gQVPPmQblkCYRc3tMRtFqoFi6EcuVKpIWFIW3YMCCP3uziNH58GqKiZNi4UWnWdu6cHAMGuGL9+hS4cL4xIiIiu3J75RV7l5Cj5D17inT8m2++ia+++go//PADxo0bB0EwHQ2XEa4zeqEvX76MadOmoXXr1ujcuTNcXV1x+fJlbN68Gb///jv2798Pf3//QtWSkpKCoKAgREZGomnTpmjWrBnu3r2Ld999F+3bt7d4zMcff4z69eujXbt28PHxQUxMDHbu3ImuXbtizZo1CAoKAgC0atUKUVFRWL9+PVq2bIlWrVplnsPLyyvXuhYsWIAJEyagVKlS6N69O1xdXbFr1y5MmDABR44cwdq1a81+blFRUejYsSPq1KmDt956C9evX8fOnTsRHByMY8eOoVy5coX6GVmjxt9++w29e/eGl5cXOnXqhPLly+PBgwc4d+4cNm7ciHfeeQcAcObMGbz22muQy+Xo1KkT/Pz8EB8fj4sXL2LlypUYM2aMVb+HvDBEE1mRVLo0tJMnI+2DD6CaNQvKlSsh6HQ57i8kJ0M9cyZUS5cibfhwpA0enD59to0JAjB/fipu35bh4EHzx8Iff7hgzBgN5s5NhcDR3URERHajOH7c3iUUm8qVK6N9+/b4448/EBERgbZt22a2GQwGbNq0CSqVKnMS4Fq1auHSpUsoVaqUyXkiIiLQtWtXzJw5E99++22hapk3bx4iIyPx9ttvY86cOdDpdJnL0oaGhlo85ujRo2ZzMN29excvv/wyvvjii8wQ3bp1awDA+vXr0apVK3z66af5qun69euYNGkSypYti7/++ivzfenPP/8cXbt2xY4dO7Bx40azpXMPHTqESZMmYeTIkZnbpk6dipkzZ2LdunUYNWpUvq5fHDWuXbsWkiRh27ZtZqMBHj16lPn3jRs3Ii0tDevWrcv8OVraz1Y4nJuoGEi+vtB+/TUST5yA7u23IT1ZkzwnQnw81FOnwqNxYygXLADsMNukUgmsXZuM2rUtryG9apUSc+dyDWkiIiIqPhm9zBnvPmfYvXs37t27h06dOmWGZi8vL7MADaS/X12nTp18LVubkw0bNkCpVGL8+PEm2zt06GAS7rOyNIlx+fLlERwcjKtXryIqKqrQ9QDAjz/+CIPBgGHDhplMOKZSqTBp0iQAwA8//GB2XJUqVfDhhx+abMv4OZ88ebJINVmrRkuTxJUuXbrQ+xU3hmiiYiT5+SF13jwkHT8OXc+ekPLoxpU9fAjNhAnwaNIEymXLgFx6sYuDtzewaVMyypWzPCP35Mlq/PQTx3QTERFR8ejUqRN8fHywfft2xMfHZ27PaUKxAwcO4M0330Tt2rXh4+OT+W5xZGQk7t69W6gaEhIScPPmTVSrVs3iMrrNmze3eNyNGzfw4YcfonHjxvD19c2sZcmSJQBQ6HoynDlzBgBMhn9nePHFF6FWqy2+59ygQQPIZKaxr9KT5Vqz/oytoaA1ZvTqd+zYER9//DG2bduGhw8fmh3brVs3yGQyvPXWWxg6dCg2b96MmJgYq9ZeEAzRRDYgVquG1CVLkHT4MPRvvJHn/rK7d6EZMwYeL7wAl3XrAAuTMBSXKlUkbNyYAldXy2tIh4VpcPhw7j3rRERERIXh4uKCXr16ITU1NXNp2NjYWPzxxx+oXLky2rVrl7nvL7/8gjfeeAMHDhxAs2bNEBYWhrFjx2LcuHHw8/ODrpCdEYlPZlT18fGx2G7pHeJr166hXbt2WLduHQICAjBgwAB8/PHHGDduHFq2bAkASEtLK1Q92esqW7asWZsgCChbtmzmPll5WHhVUKFIf33PaLQ8AtFWNXbt2hXr1q1DYGAgVqxYgX79+qFGjRp44403MgM5ALzwwgvYvn07WrRogc2bN+O9995DYGAg2rdvbzZxmy3wnWgiGxLr1kXK6tWQnToF9bRpcPn991z3l0VFwXXoUBjnzEHap59C360bICv+z76aNDFi2bIUvPWWK0TRtPdcpxPQt68rfv89GTVrcg1pIiIiWzI0bWrvEopdv379sHDhQqxZswYDBw7Exo0bYTAY0LdvX5Me1a+++gpqtRr79u1D9erVTc7x888/F/r6GaHzQQ6rqNy7d89s26JFixAXF4fvvvsu853tDKNGjcKhQ4cKXU/2uu7fv282YZokSbh//77FwGxLhakxKCgIQUFBSExMxN9//41t27ZhzZo16N69O44dOwbvJ5PvtmjRAi1atEBqair++ecf7N69G8uXL0evXr1w5MgRi8PpiwtDNJEdiI0bI2XTJsj//hvqqVOhsLB8QVby//6D68CBMM6aBe1nn8HQqROKe4avTp0M+OorLcaONX/35PFjGXr0cMWePckoW9ZyjzURERFZX1FnwHYGderUQdOmTXH8+HGcO3cO69aty1wCK6vr16+jTp06ZgH67t27uHHjRqGv7+npiSpVquDatWuIjY0161U9cuSI2TEZS2p16tTJZLskSfj777/N9pc/mS+nID3BDRs2xPbt23Hw4EE8//zzJm3//PMPtFotXnzxxXyfrzgUpUYPDw907NgRHTt2hNFoxNq1a3HixAl06NDBZD+NRoPWrVujdevW8PLywrRp0/DXX39hwIABxfZ9Zcfh3ER2ZHzpJSRv24akX3/N1yfL8shIuPXtC7cOHaDYuxeQijfADhqkw9Chloce3bghR58+rvaYA42IiIiecRnvPo8ZMwaXLl1Cu3btzHo2/fz8cP36dZOeYa1Wi9GjR0Ov1xfp+r169YJOp8O0adNMtu/duxf79+8329/Pzw9A+gzdWc2ZMweRkZFm+2dMiHb79u1819SjRw8oFAosXLgQd+7cydyu0+kwceJEAOnLhNlTQWs8dOiQxQ8S7t+/DyB9QjIAOHbsGLRabZ772Qp7ookcgLFtWyS3aQPF779DPXUq5BYmhchKcfIkFCEhMDRvDu3nn8PYokWx1TZlihZRUTJs22Y+odg//ygwaJArVq5MQR4TkBMRERHlW7du3fDpp59mhtLsE4oBwKBBgzB27Fi0adMGb7zxBoxGI/766y9IkoT69evj3Llzhb7+iBEjsH37dqxatQoXLlzASy+9hLt37+KXX37Ba6+9ht9++81k/wEDBmDdunXo378/unbtitKlS+Off/7B6dOnLe5fq1YtVKhQAT///DNUKhUqVqwIQRAwaNCgHNeKrlq1KiZNmoQJEyagZcuW6NatG1xdXbF7925cuXIFnTp1MhtKbm3Lly/HH3/8YbGtf//+aN68eYFqHDduHO7evYtmzZrB398fgiDg6NGjOHHiBJo2bZo5idvcuXNx8OBBNG/eHFWqVIFarcbp06exf/9+BAQEoHPnzsX6fWfHEE3kKAQBhtdeQ9Irr0CxbRvU06ZBfulSrocojhyBe6dO0Ldvj7QJE2B87jmrlyWTAUuWpOCNN9xw/Lj5I2PbNhd88YUaX35p/ukgERERUWF4eHhkTjpVqlQps7WBAeD999+Hi4sLlixZgtWrV8PLywuvvvoqJk6ciLfffrtI13dzc8OOHTswefJkbN++HWfOnEGdOnWwYsUKJCQkmIXiRo0a4eeff8aXX36J7du3QyaT4aWXXsLu3buxa9cus/3lcjnWrFmDiRMn4qeffsqcbKtnz545hmgAGDZsGKpVq4aFCxdi06ZN0Ol0qF69OqZOnYohQ4ZAKObX/Q4fPozDhw9bbGvVqhWaN29eoBpHjx6Nbdu24dSpU9i7dy8UCgX8/f0xefJkDBw4MHPY+8CBA+Hp6YkTJ07g8OHDkCQJlStXxkcffYQPPvgAnp6exfp9ZyfExcXxhUZyCFqtFtHR0fDz84NarbZ3OfZnNMLlxx+h+uoryPP5Xo++Uydox4+HWL++1cu5f1/AK6+44cYNy13OX3+dikGDbLsklz3wPiVnwPuUnAHvU8vu379vcWZjsh9RFKHT6aBUKs2WiiLnYc3/tngXEDkquRz63r2RdPw4UubOhfhkPb/cuOzcCffWraEZOBCyK1esWk7ZshI2b05BqVKWZ+T+5BM1du7k4BYiIiIierYxRBM5OhcX6N95B4knTiB1+nSIeXyCJkgSlD/9BPeXXoJm6FAIN29arZQaNUSsW5cCpdJ8AIsoCnjvPVf8+y9fjiYiIiKiZxdDNJGzUKuhCwtD4qlT0E6cCPHJmnk5EUQRynXr4PHCC1AuWmS1Mlq0MCI83PKU3CkpAnr1csXNm8X7Pg4RERERkb0wRBM5Gzc3pI0ahcTTp6EdNw5StgXrsxP0emjGj4fL6tVWKyE0VI+JEy1PJHbvngw9e7ohLs5qlyMiIiIichgM0UTOyssLaZ9+isRTp5A2YgQkjSbX3TXjx0PI5wRl+TFyZBreecfyGtKXLsnx1ltuSLPcTERERETktBiiiZycVKYMtJMnI/Hff5E2aBAkpdLifkJSElyHDgVEyxODFZQgADNnatGxo95i+8GDCgwfroHE+f+JiIiI6BnCEE30jJDKl4f266+ReOIEdP36WdxHcegQlN99Z7VrKhTA99+noH59o8X2TZuUmDZNZbXrERERERHZG0M00TNG8vND6vz50PXvb7FdPXkyZJcvW+16Hh7Apk3JqFTJcg/3N9+osXati9WuR0RERERkTwzRRM+o1KlTIfr5mW0XtFpowsIAg8Fq16pYUcLGjcnw8LA8dnvkSA327ePSV0RERPkh8V0oIquy9n9TDNFEzypPT6TksLSV4sQJqObNs+rl6tcXsXp1ChQK84eUwSCgf383nD/PRw4REVFu1Go1tFrLK2AQUeFotVqo1WqrnY+/0RI9w4ytWyNtyBCLbaqvvoLs7FmrXu/llw2YM8fyGtIJCQJ69XLDnTtcQ5qIiCgnbm5uSEpKQmpqKnukiYpIkiSkpqYiKSkJbm5uVjuvwmpnIiKHpP3iCyj++APy//4z2S7o9XAdMgRJe/cCKutN/tWvnx43b2oxc6b5p323bqWvIb1zZxLyWN6aiIioRJLJZChTpgySk5Px4MEDe5dDAERRzOzJlMnYB+ls1Go1ypQpY9X/7RiiiZ51rq5IXbwYbq++CiHb8lby8+eh+vprpH3+uVUv+dlnaYiKkmHTJvPlts6elWPgQFf88EMKFHwCERERmZHJZPDw8IAHP3F2CFqtFgkJCfD19bXqkGByXvwohagEML7wAtJGjbLYppozB/Ljx616PUEA5s9PRcuWlicv+/13F4wdq+Ya0kRERETkdBiiiUqItLFjYaxXz2y7IIrps3WnpFj1eioVsG5dMmrVsryG9IoVKnz7rXlPNRERERGRI2OIJiopVCqkLF4MycV8zWb5f/9B/b//Wf2S3t7pa0iXLWt5DemJEzXYsoVrSBMRERGR82CIJipBxAYNkPbJJxbbVIsXQx4RYfVrBgRI2LgxBRqN5bHbQ4ZocPQo15AmIiIiIufAEE1UwqSNGAHDCy9YbHMdOhRISLD6NZ97zohly1IgCOZBOi1NQJ8+rrh6lY8jIiIiInJ8/K2VqKRRKJAaHg7JwuySsuhoaCZMKJbLBgUZMH261mLb48cydO/uigcPuIY0ERERETk2hmiiEkisWRPaiRMttilXr4bit9+K5bpDhugQFpZmse36dTnefNMVqanFcmkiIiIiIqtgiCYqoXSDB8PQqpXFNs2HH0J49KhYrjt1qhadO+stth07psDgwa4QLc9DRkRERERkdwzRRCWVTIaUhQshububN8XGQj12bLFcVi4HlixJwQsvWF5DeutWF0ycaD7UnIiIiIjIETBEE5VgUpUqSJ02zWKbcvNmKH75pViu6+oKrF+fgipVLHc5z5+vwrJlXEOaiIiIiBwPQzRRCafv1w/6V1+12KYZPRpCbGyxXLdsWQmbNyfD29tykB47Vo3duxXFcm0iIiIiosJiiCYq6QQBqfPmQfT2NmuSPXoEzciRgGR5jeeiqllTxLp1KVAqzc8vigLefdcVp07xMUVEREREjoO/nRIRpAoVoJ0502Kby65dcPnhh2K7dsuWRixaZHlK7pQUAb16uSEqiktfEREREZFjYIgmIgCAPjQUuq5dLbZpPv0UQnR0sV27e3c9vvjC8hrSsbEy9Ozphri4Yrs8EREREVG+MUQTUTpBgHbWLIhly5o3JSRAM3w4inPtqVGj0tC/v85i28WLcvTr5wad5WYiIiIiIpthiCaiTFKZMkidN89im8u+fVAuX15s1xYEYNasVHToYHkN6QMHFPjwQ01xvZ5NRERERJQvDNFEZMLQqRN0b75psU09cSJkV68W27VdXIDvv09BvXpGi+0bNijx1VeqYrs+EREREVFeGKKJyEzq9OkQK1c22y6kpEDzwQeA0XLItQZPT2DTpmRUrGh56PiMGWr88INLsV2fiIiIiCg3DNFEZM7LCykLFlhsUvz9N5Q5tFlLpUoSNm1KhoeH5bHbH36owf798mKtgYiIiIjIEoZoIrLI2K4d0t5/32Kb+ssvIYuMLNbr168vYuXKFMjl5kHaYBDQr58bIiP5CCMiIiIi2+JvoESUI+2kSTBWrWq2XdDp4DpkCIp7uuwOHQyYM8fyGtIJCQJ69nTD3btcQ5qIiIiIbIchmohy5uaG1PBwSIJ5UJWfOQPVzJnFXkL//np89JHlNaRv3ZKhVy83JCUVexlERERERAAYookoD8ZmzaD78EOLbapZsyD/999ir2HChDT06GG51/v0aTkGDnSFwVDsZRARERERMUQTUd60n34KY926ZtsFoxGaIUOAVMtDrq1FEIAFC1LRooXlpPzbby4YN07NNaSJiIiIqNgxRBNR3tRqpISHQ1IozJrkly5B/eWXxV6CSgWsW5eCmjUtL6+1fLkKCxYoi70OIiIiIirZGKKJKF/Exo2R9vHHFtuUCxdCfvhwsddQqpSEH39Mho+P5TWkP/9cg19+MQ/6RERERETWwhBNRPmWNno0DI0bm20XJAmaDz6ALWb4CgiQsHFjCjQay2O3Bw92xd9/cw1pIiIiIioeDNFElH8uLkhdvBiSSmXWJL9xA+ovvrBJGc8/b8TSpSkQBPMgnZYmoE8fV1y7xscbEREREVkff8skogIR69SBdsIEi22qFSug+PNPm9TRubMB06ZZXvrq0SMZund3xcOHXEOaiIiIiKyLIZqICkz3wQcwNG9usU0zfDgQF2eTOsLCdBgyJM1i27Vrcrz5piu0lnM2EREREVGhMEQTUcHJ5UhdtAiSm5tZkywmBppx42xWypdfahEUpLfY9vffCgwZooFoeR4yIiIiIqICY4gmokIRq1aFdsoUi23KjRuh2LbNJnXI5cDSpSl47jnLa0j/8osSkyerbVILERERET37GKKJqNB0AwZA//LLFts0o0ZBuH/fJnW4ugIbNqSgShXLXc7z5qmwYgXXkCYiIiKiomOIJqLCEwSkLlgAydPTrEn24AE0o0cDkuWlqKytXLn0NaS9vCxfb8wYNX7/nWtIExEREVHRMEQTUZFIlSoh9euvLba5bNsGlx9/tFkttWqJWLcuGUqleZAWRQEDBrji1Ck+9oiIiIio8PjbJBEVmb5XL+iDgiy2aT7+GEJMjM1qadXKiIULUy22JScL6NXLDdHRXPqKiIiIiAqHIZqIik4QkDp3LsQyZcyb4uPTl72y0bBuAOjRQ48JEyyvbRUbK0PPnm6Ij7dZOURERET0DGGIJiKrkMqWReqcORbbXP78Ey6rVtm0no8+SkO/fjqLbRcuyNG/vxt0lpuJiIiIiHLEEP0M2Lp1K7p27YqAgAB4e3vj5s2b9i6JSijDG29A17OnxTbNZ59BuHHDZrUIAjB7dirat7e8hvT+/QqMHKmxZQc5ERERET0DGKKfASkpKWjRogXGjx9v71KIkPr11xArVDDbLiQnwzUsDBAtL0NVHFxcgJUrU1CvntFi+w8/KPH11yqb1UNEREREzo8h+hnQu3dvjB07Fq1atbJ3KUSAtzdS58+32KQ4cgTK8HCbluPpCWzalIwKFSyH9+nT1Vi/3sWmNRERERGR83LaED137lx4e3vD29sbx48ft/n1N27ciJEjR6Jdu3YoV64cvL29sW7dulyPOXnyJHr06AF/f39UrFgRHTt2xJYtW2xUMZHtGDp2RNo771hsU//vf5BdumTTeipVkrBxYzLc3S2P3f7wQw3275fbtCYiIiIick5OGaIjIyMxffp0uLm52a2GqVOnYuXKlYiOjoavr2+e+0dEROC1117D0aNH0a1bNwwYMACxsbEYMGAA5ufQa0fkzLRTpkCsUsVsu5CWBs2QIYDBYNN6GjYUsXJlCuRy8yCt1wvo188NFy865SORiIiIiGzI6X5j1Ov1CAsLQ4MGDRCUw7q0OdmwYQOioqJybDcajViwYAF0+Ziyd/78+Thz5gyuXr2Kd999N9d9DQYDRowYAZlMhh07dmDevHn48ssvcfDgQdSoUQNTpkwxq2vSpEmZPe05fRE5NA8PpCxcCEkwX5NZ8e+/UOUwk3dx6tjRgNmzLa8hnZAgoEcPN8TGcg1pIiIiIsqZ04XomTNn4uLFi1iwYAHk8vwPv7x9+zZGjBiBzp07WwzSoigiLCwMEyZMwMqVK/M8X7t27eDv75+va0dEROD69evo3r07GjZsmLndy8sLo0ePhk6nw/r1602OGTZsGI4dO5brF5GjM7ZqBV1YmMU21YwZkJ0+beOKgLff1mP0aMtrSEdHy9CrlyuSk21cFBERERE5DacK0adOncKsWbMwbtw41KlTp0DHVqpUCcuWLUNMTAyCg4MRHR2d2SaKIoYMGYJNmzahd+/eeO+996xa98GDBwEA7du3N2vr0KEDAODQoUMm2318fFCrVq1cv4icgfbzz2G0cL8KBkP6bN1paTavacKENISGWh5xcuqUAgMHusJoeUJvIiIiIirhnCZEp6WlZQ7jHjFiRKHOERwcjOXLl+P27dvo3LkzoqOjM3ugN23ahJ49e2LRokWQyaz7Y7l69SoAoHr16mZtvr6+cHd3x7Vr1wp9/sePH+PMmTO4cuUKAODSpUs4c+YMHj9+XOhzElmNRoPU8HBIFkaOyCMjofrqK5uXJJMBixalonlzy+9l797tgk8+UXMNaSIiIiIy4zQhetq0abh69SoWLlxYoGHc2XXp0gVLlizBrVu3EBwcjIEDB2Ljxo0IDQ1FeHi41QM0ACQkJAAAPD09LbZ7eHhk7lMYO3fuRJs2bfD2228DAHr27Ik2bdpg586dhT4nkTUZn38eaaNGWWxTzZsH+d9/27giQKUC1q1LQY0alrucly5VYeFCpY2rIiIiIiJH5xQh+tixY5g/fz7GjBmDwMDAIp8vJCQE4eHhuHHjBrZs2YKgoCAsWbKkSOHcnvr27Yu4uDizr759+9q7NKJMaWPHwtiggdl2QRShCQuDPV5ELl1awubNyfDxsbyG9Oefq/HrrwobV0VEREREjszhQ7TBYEBYWBjq1auHUTn0ZBWUJEmIiIjI/PfFixdx7949q5zbkowe6Jx6mxMTE3PspSZ6ZiiVSFm8GJKLi1mT/No1qCdPtkNRQECAhPXrU6BWm4/dliQBgwe74tgx5/yAjYiIiIisz+FDdFJSEq5evYqzZ8+ibNmyJks8Zcxo/corr8Db2xvbt2/P83ySJGHEiBFYu3YtQkJCsGTJEty4cQPBwcG4c+dOsXwPGe9CZ7wbnVVsbCySkpJQrVq1Yrk2kSMR69VD2vjxFttUS5ZAvn+/jStK17SpEUuXpkAQzIO0ViugTx9XXL/u8I9LIiIiIrIBhx+nqFKp0K9fP4tthw8fxtWrV/H666/Dx8cnzyWnJEnCyJEjsXr1anTr1g1Lly6FXC6HTCbD4MGDERwcjO3bt6N8+fJW/R5atmyJ2bNnY+/evQgNDTVp+/PPPzP3ISoJ0oYPh2LnTiiOHzdrcx06FImHDgFeXjavKzjYgC+/1GL8eI1Z28OHMnTv7oo9e5Lh6mrz0oiIiIjIgTh8iNZoNJg/f77FtrCwMFy9ehWjR49G06ZNcz2PJEkYPXo0Vq1aZRKgAaB79+4AYBKkfX19rfY9tG3bFgEBAdi8eTMGDx6cuVZ0fHw8Zs+eDaVSid69e1vtekQOTaFAang43Fu3hpCaatIku3ULms8+Q+qCBXYpLSxMhxs3ZFiyRGXWdvWqHH37umL9+lQLRxIRERFRSeHwIdpa7ty5g+3bt6Nr165YunQpFArTb7179+6QJAlDhgzBvn370KtXr1zPt3r1ahw5cgQAEBkZCQBYs2ZN5prQzZs3R//+/QEACoUC3377LUJDQxEUFISQkBC4u7tj69atiI6OxpQpU1ClShVrf8tEDkusUQPaSZOgGTfOrE25di30QUEwvP66zesSBGD6dC2io2XYtcv83e0jRxQYMcIL48dHWziaiIiIiEqCEhOiK1asiD179qBy5cpmATpDjx490KRJE9SoUSPP8x05ciTznewMR48exdGjRzP/nRGiAaBNmzbYvXs3pk+fji1btkCv1yMwMBCTJ09GSEhIIb8rIuele/99uGzfDsWBA2ZtmhEjkPTSS5BKl7Z5XXI5sGxZCjp3dsO//5o/K379VQ1v70qYMcPmpRERERGRAxDi4uLMZ9IhsgOtVovo6Gj4+flBrVbbuxyyASEqCh4tW0JITDRr04WEIHXFCjtUle7ePQEdO7ojKsryhGLffJOA99/n45McE5+n5Ax4n5Kz4L1K2RX7dLNxcXGIjIxEWlpacV+KiJyM5O+P1GnTLLYpf/4ZLj//bOOKnipXTsKPPybDy8tyUP7kEw/8/LP5kG8iIiIierYVOUSfPn0aX375Jfbu3WuyPTU1FQMHDkS1atXQqlUr1KlTB7/++mtRL0dEzxj9W29B/9prFtvUH30E4e5dG1f0VO3aItauTYaLi3mQNhoFvPeeBj/+yCBNREREVJIUOUSvXbsWs2bNgiSZ/pI5bdo0/Pzzz5AkCZIkIS4uDu+//37mJFxERAAAQUDqvHkQS5Uya5I9fgzNiBGAZL9h061bG7FggeUZuUVRwODBGmzYwCBNREREVFIUOUQfPnwYarUaL7/8cuY2nU6HVatWwcXFBZs2bcKNGzcwePBg6PV6LF68uKiXJKJnjFS+PLSzZllsc/ntN7isXWvjikz16qXHZ59pLbaJooCwMA3WrWOQJiIiIioJihyi7927hwoVKkAme3qqY8eOITExEa+//jpeeeUVeHl5YeLEiXBzc8OhQ4eKekkiegbpQ0Kg69bNYptm/HgIUVE2rsjUmDFpGDHC8twOkiRg2DANVq9mkCYiIiJ61hU5RMfFxaFUtmGYx44dgyAI6NChQ+Y2jUaDgIAAxMTEFPWSRPSM0s6aBbFcObPtQmIiXIcNA0TRDlU9qUEAJk3SYsSIJIvtkiTgww9d8f33ShtXRkRERES2VOQQrdFo8ODBA5NtR44cAQC89NJLJtuVSqVJjzURUVZS6dJInTfPYpsiIgLKpUttXJEpQQA++SQZ772X84eBo0ZpsHQpgzQRERHRs6rIibZWrVqIiorChQsXAAAPHz7EgQMHUKZMGdSuXdtk3zt37sDHx6eolySiZ5jh9deh69vXYpt60iTI/vvPxhWZEgRg8OAYfPyx5R5pAPj4Yw3CwxmkiYiIiJ5FRQ7RXbt2hSRJ6NGjBz777DMEBwdDp9MhJCTEZL/o6GjcvXsX1apVK+oliegZlzp9OsTKlc22C6mp0ISFAUajHaoy9dFHyfj8c8uTjQHAp59qsGABgzQRERHRs6bIIXrQoEFo0aIFbt++jUWLFuHChQuoUaMGxo0bZ7Lfli1bAACtW7cu6iWJ6Fnn6YmUhQstNimOH4fq229tXJBlH32UhsmTLS9/BQATJmgwbx6DNBEREdGzRFHUEyiVSmzbtg27du3ClStX4Ofnh6CgIKjVapP95HI5hgwZgi5duhT1kkRUAhjbtkXa++9DZeE9aNW0adC/+irEevXsUJmpESN0kMvTA7MlEydqYDAI+OgjyzN7ExEREZFzKXKIBgCZTIagoKBc9xk6dKg1LkVEJYh28mQo9u6F/OpVk+2CXg/XIUOQ9OefgNL+Pb3DhumgUACffGI5SE+ZoobBAIwbxyBNRERE5Ow4VTYROS5XV6QuWgTJwqz+8rNnofr6azsUZdmQITp8803OQ7unT1fjyy9VkCQbFkVEREREVlfkEP3gwQPs378f/1mYMff7779Hy5YtUa1aNfTo0QNXrlwp6uWIqIQxvvQS0j780GKbas4cyE+csHFFOXv/fR3mzMk5SH/zjRpTpjBIExERETmzIofoxYsXo1u3bjh+/LjJ9pUrV+Kjjz5CZGQkHj9+jD/++APBwcF49OhRUS9JRCVM2qefwhgYaLZdMBrTZ+tOzTm42tqAATp8+20KBMFyUp49W42JE9UM0kREREROqsgh+sCBA5DL5QgODjbZPnv2bADA8OHDsXbtWjRv3hz37t3DokWLinpJIippVCqkhIdDUphP4yC/fBnqKVPsUFTO+vfXY8GC1ByD9LffqvDZZwzSRERERM6oyCE6Ojoavr6+cHd3z9x29uxZREdH46WXXsL//vc/BAUF4fvvv4dcLsdvv/1W1EsSUQkkNmqEtLFjLbYpw8MhP3jQxhXlrm9fPRYvToVMZjkpL1qkwrhxDNJEREREzqbIIfrRo0coX768ybajR48CADp16pS5zdfXF9WqVcONGzeKekkiKqHSRo+G4bnnzLYLkgTXoUOBxEQ7VJWzXr30WLIk5yC9ZIkKY8aoIYo2LoyIiIiICq3IIVomkyEpKclk27FjxyAIApo1a2ay3dPTEzqdrqiXJKKSSqFAang4JJXKrEl28ybUn39uh6Jy1727HsuXp0Iutxykly9XYfRoBmkiIiIiZ1HkEO3v749r167h8ePHAAC9Xo+9e/dCo9GgSZMmJvs+fPgQZcqUKeoliagEE2vXhjaHsKxauRKKP/6wcUV569ZNjxUrUqBQWA7SK1eq8OGHGgZpIiIiIidQ5BDdvn176PV6DBw4ELt27cLw4cPx6NEjdOjQAYoskwDFx8fjxo0bqFSpUlEvSUQlnO6DD2Bo0cJim2b4cCAuzrYF5UOXLgasWpUCFxfLQXrtWiWGDtXAaLRxYURERERUIEUO0SNHjoSvry/++usv9O3bFxs3boRarca4ceNM9tu9ezckSULz5s2LekkiKulkMqQsWgTJzc286c4daHKYgMzegoIMWLMmBUql5SC9fr0SYWEaGAw2LoyIiIiI8q3IIbpcuXLYu3cvBg8ejPbt2+Odd97Bvn37UK9ePZP9jhw5gvr16+O1114r6iWJiCAFBEA7darFNuWmTVD8+quNK8qf//s/A9auTYFKZTlIb9qkxODBDNJEREREjkqIi4vjAivkELRaLaKjo+Hn5we1Wm3vcsgZSBJcu3eHy59/mjWJZcog6ehRSGXLWvWS1rpP//xTgb59XaHVChbbu3bVYenSVLi4FPoSVILxeUrOgPcpOQveq5RdkXuiiYjsRhCQOn8+JC8vsybZw4fQjBwJR12IuUMHAzZuTIZGY7m+X35RYsAAV3BBAyIiIiLHosh7l/y7d+8e/vrrL1y5cgWJiYnw8PBArVq18PLLL6OslXuDiIgAQKpYEalffw3XwYPN2lx27IDLxo3Q9+5th8ry1ratEZs2JaNXLzekpJj3SG/f7oK333bFypUpsLCqFxERERHZgVVCdFpaGj7//HOsWrUKer3erN3FxQUDBgzA5MmToeJvgkRkZfqePaHfvh0u27aZtWnGjoWhVStIlSvbobK8tW5txObNyejRww3JyeZBetcuF/Tv74pVq1LAEWRERERE9lfk4dyiKKJPnz5YtmwZdDodfHx80Lx5c3Tr1g3NmzeHj48PdDodlixZgr59+0Jy0KGVROTEBAGpc+ZA9PExb0pISF/2yoGfPS1aGPHTT8nw8LBc42+/ueCtt1yh1dq4MCIiIiIyU+QQvXbtWvz111/w8PDAt99+i8jISOzYsQPLly/Hjh07cOHCBcyfPx9eXl7Yu3cv1q1bZ426iYhMSD4+SJ0712Kby19/Qfn997YtqICaNTPi55+T4elpOUj/8YcL+vRxRWqqjQsjIiIiIhNFDtEbN26EIAhYvXo1+vXrB4XCdIS4XC7HW2+9hZUrV0KSJKxfv76olyQissjQuTN0vXpZbFN//jlk16/buKKCadrUiF9+SYaXl+Ug/ddfLk/en7ZxYURERESUqcgh+vz586hSpQratm2b635t27ZFQEAAzp8/X9RLEhHlKHXGDIgVK5ptF5KTofngA8BotENV+ffcc0b8+msSvL1Fi+0REQr06OGGpCQbF0ZEREREAKwQolNTU1GqVKl87VuqVClo+VIfERUnb2+kLlhgsUlx5AiUixbZuKCCa9xYxNatyShd2nKQPnQoPUgnJtq4MCIiIiIqeoj29fXFlStXkJrHi3opKSm4cuUKypUrV9RLEhHlytC+PdLefddim3rqVMguXLBxRQXXsGF6kC5TxnKQPnJEgdBQNyQk2LgwIiIiohKuyCG6devWSE5Oxvjx43Pdb/z48UhOTkabNm2Kekkiojxp//c/GAMCzLYLaWnQhIUBFpbjczT164vYti0ZZctaDtLHjikQEuKGuDjb1kVERERUkhU5RI8YMQIuLi5YtWoVWrVqhbVr1+LkyZO4desWTp48ibVr16Jly5ZYvXo1lEolPvzwQ2vUTUSUO3d3pC5aBEkwX3tZceoUVLNn26GoggsMFLF9ezJ8fS0H6X/+UaBbNwZpIiIiIlspcoiuVasWFi9eDJVKhfPnz+PDDz9Ex44d0bBhQ3Ts2BEffvghIiMjoVKpsHjxYtSqVcsadRMR5cnYogV0Q4dabFN98w1kp07ZtqBCql07PUiXL285SP/7rwJvvOGOR4/MPzAgIiIiIusqcogGgG7duiEiIgJ9+/ZFuXLlIElS5le5cuXQr18/REREoGvXrta4HBFRvmknTICxdm2z7YLBANewMMBJJjusWVPEjh3JqFjRcpA+c0aON95ww8OHDNJERERExUmR9y75U7NmTSx4MiNuQkICkpKS4O7uDk9Pz8x92rZti/j4eJxykt4fInoGqNVIXbwYbh07Qsi2vJX8wgWop0+HdvJkOxVXMNWrpwfp4GA33Lpl/hnouXNyBAe74ddfk1G2rOW1pomIiIioaKzSE52dp6cnKlasaBKgAeDWrVuIiooqjksSEeXI2KQJ0j76yGKb8ttvIT961MYVFV7VqiK2b0+Cn5/lHunIyPQgfe8ee6SJiIiIikOxhGgiIkeTNmYMjA0bmm0XJCl9tu7kZDtUVTgBARJ27EhClSqWg/TFi3J07uyGu3cZpImIiIisjSH6GbB161Z07doVAQEB8Pb2xs2bN+1dEpHjUSqRsngxJKXSrEl+/TrUkybZvqYi8PdPD9JVqxottl++nB6kY2IYpImIiIisiSH6GZCSkoIWLVrkuVY3UUknBgZC+9lnFttUS5dCvm+fTespqsqVJezYkYzq1S0H6f/+kyMoyA23bjFIExEREVkLQ/QzoHfv3hg7dixatWpl71KIHJ5u2DAYXnrJYpvr0KFAfLyNKyqaihXTg3StWpaD9PXrcgQFuSMqikGaiIiIyBqcIkRrtVqMHz8er7/+OurUqQNfX1/UqlULr732GtauXQu9Xm/zmjZu3IiRI0eiXbt2KFeuHLy9vbFu3bpcjzl58iR69OgBf39/VKxYER07dsSWLVtsVDERAQDkcqSGh0NydTVrkt2+Dc2nn9qhqKIpX17Ctm3JqFPHcpC+eVOGoCB33LjBIE1ERERUVE4RopOTk7FixQoIgoBXX30VQ4cORefOnRETE4Nhw4ahV69eEEXLE+wUl6lTp2LlypWIjo6Gr69vnvtHRETgtddew9GjR9GtWzcMGDAAsbGxGDBgAObPn2+Diokog1itWo7LWil/+AGKnTttXFHR+fqmB+nAQMtBOjpahs6d3XH9ulM89omIiIgcVoHXiZ4xY0ahL5aamlqo40qVKoWoqCgos00IZDAY0LVrV+zduxd79uzBa6+9lut5NmzYgBYtWsDf399iu9FoRHh4OAYNGmR2rezmz5+PatWqwd/fH3PmzMHkXNaZNRgMGDFiBGQyGXbs2IGGT2YIHjt2LDp06IApU6agS5cuJnVNmjQJc+fOzbWGuLi4XNuJKGe6gQPhsn07FPv3m7VpRoxA0ksvQSpTxg6VFV7ZsulBuksXN5w7Jzdrv3VLhs6d3bB1azKqV7ftB49EREREz4oCh+ivvvoKglC4IYGSJBXqWJlMZjHUKhQKdO7cGQcPHsS1a9dyPcft27cxYsQI+Pr6Yvv27WZBWhRFhIWFYdOmTVAqlRg0aFCu52vXrl2+64+IiMD169fRt2/fzAANAF5eXhg9ejQ++OADrF+/HuPGjctsGzZsGN588818X4OICkgmQ8qCBfBo2RJCQoJp0/370IwejZSVK4FCPu/spUwZCVu3JqNrVzecOWMepG/ffhqka9ZkkCYiIiIqqAKH6BYtWhQ6RFubKIr4888/AQCBgYG57lupUiUsW7YMAwYMQHBwMLZv3w4/P7/M8wwZMgSbNm1C79698d5771m1zoMHDwIA2rdvb9bWoUMHAMChQ4dMtvv4+MDHx8eqdRCRKcnPD6nTp6dPKJaNy6+/wuWnn6Dv3t0OlRVN6dIStm5NQrdubvj3X/PH/J07T4N07doM0kREREQFUeAQvWPHjuKoI190Oh1mzZoFSZLw+PFj7N+/H5cvX0bfvn3Rtm3bPI8PDg7G8uXLMXDgQHTu3Bnbt29HpUqVMnuge/bsiUWLFkEms+47g1evXgUAVK9e3azN19cX7u7uefak5+bx48eIjo7G9evXAQCXLl1CfHw8/Pz8UKpUqUKfl6gk0L/5JvTbtsFl926zNvWYMTC0bAmpQgU7VFY03t7Ali3J6N7dDf/8Y/6oj41ND9K//pqMwEAGaSIiIqL8KnCItiedTmfyTrYgCBg+fDgmTpyY73N06dIFRqMR77//PoKDg9GkSRNs2bIFoaGhCA8Pt3qABoCEJ0NFPT09LbZ7eHhk7lMYO3fuxNAsPWk9e/YEACxcuBB9+/Yt9HmJSgRBQOq8eZA3bw7Zo0cmTbK4OGhGjEDKxo1ON6wbSA/SP/+cjB493PD33+aP+/v3ZQgOTg/S9eszSBMRERHlh1NN0+ru7o64uDg8evQI58+fx8yZM7F69Wp07ty5QCE0JCQE4eHhuHHjBrZs2YKgoCAsWbIEcrn5+4POoG/fvoiLizP7YoAmyh/J1xeps2dbbHP5/Xe4rFlj44qsx9MT2Lw5Gc2bGyy2P3yYHqRPn3aq/zsgIiIishun/K1JJpOhUqVKGDhwIObNm4ejR49i1qxZ+T5ekiRERERk/vvixYu4d+9ecZQK4GkPdE5BPzExMcdeaiKyDUPXrtCFhlps04wfD+HmTRtXZD0eHsCPPyajZUvLQfrxYxm6dHHDqVNO+X8JRERERDbl9L8xvfzyywCeTt6VF0mSMGLECKxduxYhISFYsmQJbty4geDgYNy5c6dYasx4Fzrj3eisYmNjkZSUhGrVqhXLtYko/7QzZ0K0sO67kJSUPvmYjdejtyZ3d2DTpmS0aWM5SMfFyfDGG+44ccI5R+QQERER2YrTh+i7d+8CAFxcXPLcV5IkjBw5EqtXr0a3bt2wdOlS9OzZE9999x2uX7+O4ODgzPNZU8uWLQEAe/fuNWvLmF08Yx8ish+pVCmkfvutxTbFwYNQfvedjSuyLjc3YMOGZLz8st5ie0KCgG7d3HD8OIM0ERERUU6cIkRfvHgRKSkpZttTUlLw2WefAQBeeeWVXM8hSRJGjx6NVatWZQbojHegu3fvbhKkY2NjrVp/27ZtERAQgM2bN+PMmTOZ2+Pj4zF79mwolUr07t3bqtckosIxvPYadP36WWxTT54M+X//2bgi63J1BX74IQUdO+YcpENC3HD0KIM0ERERkSVOMTv3li1bsGjRIjRr1gz+/v7w8PBATEwM/vjjDzx69AjNmzfHBx98kOs57ty5g+3bt6Nr165YunQpFArTb7179+6QJAlDhgzBvn370KtXr1zPt3r1ahw5cgQAEBkZCQBYs2ZN5rDy5s2bo3///gAAhUKBb7/9FqGhoQgKCkJISAjc3d2xdetWREdHY8qUKahSpUqhfjZEZH2pX34Jxb59kEVHm2wXtFp4fvghsHChnSqzDo0GWLs2BW+/7YrffjMfxZOYKCA01A2bNiWjZUujHSokIiIiclxCXFycZO8i8vLvv/9i5cqVOHbsGGJiYpCcnAxPT0/Uq1cPoaGheOutt8xCsSU3btxA5cqVc933v//+Q40aNfI8V1hYGNavX59je58+fRAeHm6y7cSJE5g+fTqOHTsGvV6PwMBADB06FCEhIXleryTQarWIjo6Gn58f1Gq1vcuhEk4eEQH3N96w2Hbrgw/g8sUXTn+fpqUB77zjil27LL8O4+oqYePGZLRuzSDtbPg8JWfA+5ScBe9Vys4pQjSVDHxAkaNRjxsHlYX3oEWFAo9/+w2K55+3Q1XWpdMB777riu3bLQdpjUbChg3JaNuWQdqZ8HlKzoD3KTkL3quUnVO8E01EZA/aiRNhtDAyRWYwwGvwYCh27waMzh0ulUrg++9T0KWL5XekU1MF9Orlhr17neLtHyIiIqJixxBNRJQTV1ekhodDkpk/KhX//Qe33r3h0bgxVLNmQSjGteaLm4sLsHx5CkJDdRbbtVoBffq4Ys8eBmkiIiIihmgiolwYmzZF2siRObbLoqOhnjIFHvXqQfPuu5AfOABIzveWjEIBfPddKnr2tByk09IE9O3rit27GaSJiIioZGOIJiLKQ9q4cTDWq5frPoJeD+XPP8M9OBjuzZpBuXgxEBdnmwKtRKEAwsNT0bu35SCt0wno188VO3YwSBMREVHJxRBNRJQXlQopq1fDWL16vnaXX7oEzSefwDMwEJrhwyE7dap467MiuRxYuDAVb71lOUjr9QLeftsVW7cySBMREVHJxBBNRJQPYvXqSPrzTyR98gnSypfP1zFCSgqUa9bAo107uLVvD5e1a4GUlGKutOjkcuDbb1Px9tuWg7TBIGDAAFf88guDNBEREZU8DNFERPnl7Y3kkSNx9pdf8Hj1auhfeQWSIOTrUMXJk3AdNgyedetC/emnkF2+XMzFFo1MBsyZk4qBA9MsthuNAgYOdMXmzZaXxiIiIiJ6VjFEExEVlFwO3auvIuXHH5H477/QjhwJsUyZfB0qxMdDFR4OjxdfhFtwMBS//ALoLS8vZW8yGTBzphaDBuUcpAcN0mDjRgZpIiIiKjkYoomIikAKCEDapElIjIxEytKlMDRvnu9jFQcOwO2dd+BRvz5UU6dCuHWrGCstHEEAZszQIizMcpAWRQFDhmjwww8M0kRERFQyMEQTEVmDSgV9jx5I3rULiYcPI+399yF5eOTrUFlsLNQzZ8KjYUO49ukDxR9/AKJYzAXnnyAA06ZpMXy45SAtSQKGDtVg9WoGaSIiInr2MUQTEVmZGBgI7TffIOHCBaTMnQtjgwb5Ok4QRbjs2gW37t3h/txzUM6bB+HBg2KuNn8EAfjf/7QYNUprsV2SBHz4oSu+/15p48qIiIiIbIshmoiouLi7Q//OO0iKiEDSnj3Q9ekDSa3O16HyGzegmTgRHoGB0Lz/PuRHjgCSVMwF504QgC++SMOYMZaDNACMGqXB0qUM0kRERPTsYogmIipuggBj06ZIDQ9H4oULSJ06Nd9rTgs6HZQ//gj311+He8uWUC5bBiQkFHPBudQjABMmpOGTT3IO0h9/rEF4OIM0ERERPZsYoomIbEgqVQq6YcOQdPw4kn75BfrgYEhyeb6OlUdGQjNmDDwDA6EeNQqys2eLudqcffJJGiZMyDlIf/qpBgsWMEgTERHRs4chmojIHmQyGNu1Q8qaNUg8exbaTz6BWLFivg4VkpKg+v57eLRuDbdXX4XLhg2ANudAW1zGjEnDpEmpObZPmKDBvHkM0kRERPRsYYgmIrIzqWJFpH3yCRLPnEHymjXQt2+f72MVx47BdcgQeNStC/WECZBdu1aMlZobOVKHKVNyDtITJ2owa5bKhhURERERFS+GaCIiR6FQwBAcjJSff0biyZNIGz4cYqlS+TpU9vgxVAsWwOO55+DarRsU27YBBkMxF5xu+HAdpk/POUhPmaLGjBkM0kRERPRsYIgmInJAYrVq0E6ZgsQLF5CyeDEML76Y72Nd/voLbv36waNhQ6i++gpCTEwxVpouLEyHb77JOUhPn67Gl1+q7D3BOBEREVGRMUQTETkytRr63r2R/PvvSDxwAGkDBkByc8vXobKYGKi/+goeDRrAtV8/KP76CxDFYiv1/fd1mD075yD9zTdqTJ3KIE1ERETOjSGaiMhJiA0aQDtnDhIuXEDqzJkwBgbm6zjBaITLtm1w69YN7k2bQjl/PoRHj4qlxnff1eHbb1MgCJaT8qxZakyapGaQJiIiIqfFEE1E5Gw8PaF77z0kHTqEpN27oevZE5Iyf7Ngy69ehebzz+FRty40Q4ZAfvw4rJ1o+/fXY/781ByD9Lx5KkyYwCBNREREzokhmojIWQkCjM2aIXXJEiRGRiJ18mSIVark79C0NCg3bID7K6/AvU0bKL//HkhKslppb72lR3h4KmQyy0l54UIVPvmEQZqIiIicD0M0EdEzQPLxgW7ECCT++y+SN2+G/vXXIcny94iXnz0LzahR8KxbF+oxYyCLjLRKTb176/HddzkH6e++U+Hjj9XF+Zo2ERERkdUxRBMRPUtkMhg6dkTK+vVIPH0a2jFjIPr65utQITERqmXL4NGiBdxefx0uP/4IpKUVqZwePfRYtiwVcrnlIL1smQoffcQgTURERM6DIZqI6Bkl+fkhbcIEJJ47h+RVq2Bo0ybfxyqOHIHr++/Do149qCZNgnDjRqHrCAnRY8WKFCgUloP099+rMGKEhkGaiIiInAJDNBHRs87FBYYuXZC8dSsSjx9HWlgYJC+vfB0qe/AA6rlz4dGkCVx79IBi507AaCxwCV26GLByZQpcXCwH6TVrlAgKcsOXX6qwbZsCUVEC35cmIiIihyTExcXx1xRyCFqtFtHR0fDz84NarbZ3OUQWPTP3aUoKXH7+GcoVK6A4ebJAh4qVK0P39tvQ9e8PKZ9DxTPs2qXA22+7QqcT8ty3VCkRDRuKaNTIiEaNjGjY0Ijq1UXk81XvEu2ZuU/pmcb7lJwF71XKjiGaHAYfUOQMnsX7VHbqFFTLl8Nl82YIqan5Pk5SKKDv3Bm6d9+FsXVrQMg7GAPA778r0K+fK9LS8rd/Vu7uEho0SA/UDRumh+vatUW4uBT4VM+0Z/E+pWcP71NyFrxXKTuGaHIYfECRM3im79O4OCg3boRyxQrIL10q0KHGWrWgGzAAuj59AG/vPPf/808F3nyzcEE6O5VKQmCg8UmPtYiGDY0IDDRCoynyqZ3WM32f0jOD9yk5C96rlB1DNDkMPqDIGZSI+1SSID90CMoVK+CybRsEvT7/h2o00IeGpvdOP/dcrvvu2yfHW2+5ISmp6EE6O7lcQu3aYmZvdaNGRjRoYISHh9Uv5ZBKxH1KTo/3KTkL3quUHUM0OQw+oMgZlLT7VLh3D8o1a6BcuRKy6OgCHWto3Bi6d9+Fvnt3wNXV4j737wsID1fiyBEFzp6VF0ugzqp6dWOWd6zTQ3aZMs/e/w2WtPuUnBPvU3IWvFcpO4Zochh8QJEzKLH3qdEIxR9/QLl8ORR79kAowNTZkqcndH36QPfuuxBr185xP1EErl2T4fRpOU6fluPMmfS/P35cvDOJVa5s2mPdqJER5ctL+X3F2yGV2PuUnArvU3IWvFcpO4Zochh8QJEz4H0KCDdvQrlqFZRr1kB2/36BjjW0agXdwIHQBwUBSmWe+0sSEB0t4MyZjGCd/nXnTvEG67JlTWcFb9TIiCpVnCdY8z4lZ8D7lJwF71XKjiGaHAYfUOQMeJ9modPBZdu29N7pw4cLdKhYrhx0/funL5Pl71/gS8fGmgbr06fluHmzeIO1p6dk0mPdsKERNWuKkMuL9bKFwvuUnAHvU3IWvFcpO4Zochh8QJEz4H1qmezCBShXrIBy40YICQn5Pk6SyWB45RXoBg6EoV27fPVO5yQuDmY91pcvyyBJxdd97OoqoX590yW36tYVi/JtWAXvU3IGvE/JWfBepewYoslh8AFFzoD3aR6Sk+GyeTNUy5dDfuZMgQ6VNBoYX3gBhubNYWzeHIamTQF39yKVk5QEnD//tLf69Gk5Ll6UQa8vvmDt4iKhbl3T4eD16hnh5lZslzTD+5ScAe9Tcha8Vyk7hmhyGHxAkTPgfZpPkgT5yZNQLl8Ol59/hqDVFvwUcjmMDRumB+pmzWBs3hxS2bJFLi0tDbhwQZbZW336tBznzsmRmlp8wVomk1CrlmjSY92ggTE/S2oXCu9Tcga8T8lZ8F6l7BiiyWHwAUXOgPdpwQmPH8Plhx+g/P57yP/7r0jnMtasmRmqDS1aQKpSBdaY7ctgAK5ckZn0WJ89K0dCQvHOJBYQYESjRqLJBGZlyxb9/5Z5n5Iz4H1KzoL3KmXHEE0Ogw8ocga8T4tAkiCPiIBq+XIoduyAYDQW+ZRihQrpw7+bNYOheXOIgYGw1kxfogjcvCnD6dOm4frBg+KdwKxiRRENGphOYFa5csFmBud9Ss6A9yk5C96rlB1DNDkMPqDIGfA+tQ7hzh0o16yBctUqyG7fttp5JU/P9KHfT0K18bnnAJXKeueXgDt3hCxrWad/3bpVvMG6dOmM3uqn71pXrSpClsNleZ+SM+B9Ss6C9yplxxBNDoMPKHIGvE+tzGCA/PBhKA4fhvzoUSiOH4eQnGy100sqFYzPPQdDixbpwfrFFwEvL6udP8ODB8KTQC3LDNjXrhXv2lceHukzg2ftsa5dW4RCwfuUnAPvU3IWvFcpO4Zochh8QJEz4H1azPR6yM+eTQ/WR49CfuQIZA8fWu30kkwGsV699N7qFi1gaN4cUvnyVjt/VgkJwNmzpj3WFy/KIIrF9561Wi2hXj0j6tXToVKle2jZ0hONGsnh4VFslyQqND5PyVnwXqXsGKLJYfABRc6A96mNSRJkV65AfuQIFE++ZDdvWvUSxqpVnw7/btECYvXqVpmszJLUVCAyUp5lOLgM58/LodMV7wRm/v4iAgONT77S/16zpggXl2K9LFGu+DwlZ8F7lbJjiCaHwQcUOQPep/YnxMRAceRI+vDvw4chi4yEIFnv/8rEcuXSQ/WTGcDF+vUBhcJq589OrwcuXpSZ9FifPStHcnLxBmsXFwk1a4qZwbpu3fSQ7e9fsEnMiAqLz1NyFrxXKTuGaHIYfECRM+B96oDi4qD4++/0UH3kCOQnT0LQ6ax2esndHYYXX3zaW/3CC4BGY7XzW2I0AteuyUwmMDt9Woa4uOKdwAxIf9e6Tp2nPdYZIbtMGf66QNbF5yk5C96rlB1DNDkMPqDIGfA+dQJaLeQnTmS+U634+28IiYlWO73k4gJj48bp61U/WV5LKlXKaufP8boSEBUlZC63ldFrffdu8QdrAPD1Ne21rldPRO3aRri62uTy9Azi85ScBe9Vyo4hmhwGH1DkDHifOiGjEbJz556G6iNHIIuNte4l6tZND9RPgrVUubJVz5+bu3eFzECd0XMdFWWbYC0IEqpWFTN7revVM6JuXRHVqonFOQKenhF8npKz4L1K2TFEk8PgA4qcAe/TZ4AkQXb9emaglh85AvnVq1a9hOjnZxKqxdq1i22yMkvu3ElDRMQjPHxYEVeuqBAZKceFC3IkJNimBpVKQu3aGT3WT4eGV6jA963pKT5PyVnwXqXs+DkxERGVLIIAsVo1iNWqQd+3b/qm2NjMd6oVR45AdvYsBFEs9CVk0dFQRkcDmzYBAMTSpWF86aX09aqbN4exUSMU59TYpUpJeO65JPj5pUKtTv+sXJKAW7eEJ4FahshIOc6fl+PyZRn0eusm27S0p73jWXl7i6hbV8zssQ4MNKJuXSO8va16eSIiomLFnmhyGPyUj5wB79MSIiEBin/+SV+v+sgRyE+cgKDVWu30kkYD4wsvwNC8eXqwfuEFwN3daucvyH2q1wNXr6aH6sjI9CW3LlyQ4cYNea7HWVPlyhmzgz8N1rVri1CpbFYC2QGfp+QseK9SdgzR5DD4gCJnwPu0hEpLg/z06fQh4IcPQ3H0KIT4eKudXpLLYWzY8OlkZc2bQ/LxKfT5rHGfJiUBFy+mB+v0gJ3+9wcPbPO+tVwuoUaNjFCd8c61iCpVRMhsUwIVMz5PyVnwXqXsGKLJYfABRc6A9ykBAEQRsosXM9+pVhw5Atnt21a9hLFmzcxQbWjeHFKVKvl+r7o479P794XMHuuMoeEXLsiRkmKbl51dXS0vwVWuHH+dcTZ8npKz4L1K2TFEk8PgA4qcAe9TskiSIERHm4Rq+aVLVr2EWKGC6WRlgYHIqUvW1vepKAI3b8pw/nx6oM7ovf7vPxmMRtuEax8fMcvyW+nBuk4dozVHyZOV8XlKzoL3KmXHEE0Ogw8ocga8Tym/hIcPMycrkx85Avnp0xAMBqudX/L0hKFZs6dDwJs0QcZLxI5yn6alAZcvPx0OnjGh2a1bthuPXaWK6fJbgYFG1KghFue8bpRPjnKfEuWF9yplxxBNDoMPKHIGvE+p0JKTIf/nn8wZwOXHj0NISbHa6SWVCsbnnoOhRQukPv88blSogIp16zrkfRoXB1y4IM/stc4YGh4fb5teaxcXCTVriibLb9Wta4SfH5fgsiU+T8lZ8F6l7BiiyWHwAUXOgPcpWY1eD/nZs09nAD96FLKHD616CUmthuTlBcnbO/0r4+9eXpa3Z/k7PDxsura1JAF37gjZZgmX49IlGdLSbFOHp6eEunWNJjOF16snolQp/qpUHPg8JWfBe5WyY4gmh8EHFDkD3qdUbCQJsitXTGYAl928ab9yZDLTcJ0tbCO3EO7lBSgUVqnDYACuXZPhwoWnPdaRkTJcvy6DJNkmXJcvL2ZOYFajhhHVqomoXl1EhQoSZwovAj5PyVnwXqXsGKLJYfABRc6A9ynZknD7NhRHjz6dATwyEoLkHP+3LXl4mPZ45xLIs/8dGk2e509JAS5dkuP8eZnJ+9axsbZLtRqNhKpVxcxQXb16esCuVi09YHNoeO74PCVnwXuVsrPOx8RERERkdVKlStCHhkIfGpq+IS4Oir//Tg/VR49CfvIkBJ3OvkXmQEhMhJCYCNy6VeBjJZUq56D9JJi7eHujqbc3XqjiDanh0/0e6jxxPjLjfev0XusLF+RISrJ+ok1NFTInTcvO1TU9YGcP19Wri/D1ZcAmInJmDNFERETOwtsbhtdeg+G115AGAKmpkJ88mflOteLvv9ODq5MT0tIgxMYCsbEFPtZTJkOAlxc6ZQ3eHbyRqPDCPV1p3EouheuPS+HSvTKIvFMaD8VSeIz0rzh4w2ilX41SUgScPy/H+fNyAKZTgbu7ZwRs03BdvbqIsmUZsImIHB1DNBERkbPSaGBs2RLGli3T/200QnbuHKSDB5F86RK8RBEuiYkQ4uKA+HgIcXHpX/HxTjMsvKAEUYTw+DHw+LHJ9jJPvurmcXyS4I5H0tNQnTVgP0QZXEYtnEc9XEFN6KEsVI1JSQLOnpXj7FnzHmwPD+lJsDaienXTgF2mDAM2EZEjYIgmIiJ6VsjlEBs1grZ2bdyOjoYsp/f3RBF4Eq6F7OE648+s27Pv56BDyK3BXUqCO5Lgj+hc99NDkRmoIxGI86iXGa4NKPwi1ImJAk6fluP0afOA7ekpmYXrjD9Ll2bAJiKyFYZoIiKikkYmS59d28sLBe6PliRAq7UcrnP6e1wchISE9D+TkorjO7I5FxhQD5Goh0iT7Tq4ZIbrjK9IBOI/1ChSuAaAhAQBp04pcOqUeZuXl4Tq1Z8G7Kw92Fyii4jIuhiiiYiIKP8EAdBoIGk0kCpUKPjxev3TQJ1H8DYbgh4fD0EUrf4tWZMSetTHedTHeZPtOrjgEmqbhOvzqIerqG6V97Dj4wWcPKnAyZPmbaVKiRbDdbVqRnh7F/nSREQlDkP0M2Dr1q1YsWIFTp06hbi4OJw+fRpVqlSxd1lERETmXFwglSkDqUyZgh+bMQw9t97v3HrD7TgMXQk9GuAcGuCcyfY0KHMM1yLMh3QXxuPHMvzzjwz//GPeVqaMebiuXt2IqlVFeHlZ5fJERM8chuhnQEpKClq0aIFOnTph7Nix9i6HiIioeGQdhu7vX/DjU1MLNgQ9aygvpmHoKujQEGfREGdNtmuhwkXUMXnf+jzq4RqqWS1cA8DDhzI8fCjD8ePmbT4+otnkZlWrpg8Z9/CwWglERE6HIfoZ0Lt3bwBAZGRkHnsSERGVYEUdhn77NuQXLkB+8SJkGX9evgxBq7V6qWqkoTFOozFOm2zXCmpcQh2cleqZTGp2HVWtGq4B4MEDGR48kOHvv83bypXLPkT86XJd7u5WLYOIyOE4RYiOiYnBL7/8gj179uDKlSuIjY1FqVKl8NJLL2HEiBF44YUXbF7Txo0bceTIEZw6dQqRkZHQ6XRYuHAh+vbtm+MxJ0+exPTp0/H333/DYDAgMDAQQ4cORbdu3WxYORERERWYiwukgAAYAgJgeP31p9uNRshu3kwP1ZcuQXbxIuQXLqSH67Q0q5ehlrRohFNohFMm29NkalxV1sUpXT2cEZ8G7OuoCgkyq9dx754M9+7JcOSIeVv58uKTdbCfvnud3ostws3N6qUQEdmcU4ToJUuWYO7cuahatSpefvll+Pj44OrVq9ixYwd27NiBZcuWISQkxKY1TZ06FdHR0ShTpgx8fX0RHZ37UhgREREIDQ2FWq1GSEgI3N3dsXXrVgwYMAC3bt3C8OHDbVQ5ERERWY1cDrFaNYjVqsEQFPR0u9EI2Y0bT3usL15M//PKlWIJ1ypRi0DtvwjEv3gzy3adQoMo1zo4j3o4llwPp431cR71cBNViiVcA8DduzLcvWs5YFeo8HR4uL+/DB4e3qhb1wXly8tQurQEb28JCqf47ZSISjIhLi7O4dc92Lp1K0qXLo1WrVqZbD98+DC6dOkCNzc3XLp0CSqVKtfzbNiwAS1atIB/Du9RGY1GhIeHY9CgQVAqlbmea9++fahWrRr8/f0xZ84cTJ48OceeaIPBgKZNmyImJgZ79uxBw4YNAQDx8fHo0KEDoqKi8M8//5jUNWnSJMydOzfXGuLi4kz+HRkZiRYtWjjtxGJarRbR0dHwy2ldUyIHwPuUnAHvUwdmMJiH6wsXIPvvP5tOfKZXuuKOVx1cdgnEv7p6OBRXH6cM9REF/2IL1/nl5SWhdOn0ta9Ll5ZQqpSU+fenX6LJv11d7VoyPeP4TKXsnOKzvjfeeMPi9hYtWqB169bYu3cvIiMj0aRJkxzPcfv2bYwYMQK+vr7Yvn27WZAWRRFhYWHYtGkTlEolBg0alGtN7dq1y3f9ERERuH79Ovr27ZsZoAHAy8sLo0ePxgcffID169dj3LhxmW3Dhg3Dm2++ael0RERE5KwUCog1akCsUQOG4OCn2w0GyK5ft9xzrddbvQwXXQr875+EP06iI4CPM8pQu+F+2Tq44RqI81Ig/k6qh7/uNcBVQxUAgtXrsCQ+XkB8vBzXr+f/GLXaUuAWcwnh6b3eMvt+XkBETsopQnRuXFxcAAByee6TaVSqVAnLli3DgAEDEBwcjO3bt8PPzw9AeoAeMmQINm3ahN69e+O9996zao0HDx4EALRv396srUOHDgCAQ4cOmWz38fGBj4+PVesgIiIiB6VQQKxZE2LNmjBk7TzQ6y2H6//+K5ZwrdAmo0L0CVTACTQHkPEbkdHVHfGV6uC2d11cUtTDv9pARDysj6O3q8BgtH8S1WoFxMQIiInJ/zEyWXqQzqvHO/t2dkQSkVOH6OjoaOzbtw/ly5dHvXr18tw/ODgYy5cvx8CBA9G5c2ds374dlSpVyuyB7tmzJxYtWgSZlT+WvHr1KgCgevXqZm2+vr5wd3fHtWvXCn3+x48fIzo6GteffGR76dIlxMfHw8/PD6VKlSr0eYmIiMjOXFwg1qoFsVYtGLp0ebpdr4fs6tWnw8EvXXoarg0Gq5chT0lC6Sv/oDT+QQMA3Z9sF909kBpQG/fKBeK6JhBnxXo4mlgPf9/yQ1S0HEajbXqvC0MUBTx6JODRo4Id5+r6NHSXKZPXsHMJpUqlr7ktOO6PgogKyGlDtF6vx+DBg5GWloZJkybl2ROdoUuXLjAajXj//fcRHByMJk2aYMuWLQgNDUV4eLjVAzQAJCQkAAA8PT0ttnt4eGTuUxg7d+7E0KFDM//ds2dPAMhztnAiIiJyUi4uEOvUgVinDgxduz7drtNBdvWq6TJcly6lh2uj0eplyJIS4XbuH1TFP6gKoD2AEQAkT08YmtRGfKW6uOUViIvyQJxMq4eTdyvj6jU5oqJkEEXnTJUpKQJSUgTcugUgn8uKyeWmITvj72XK5DzsvFQpCXlM0UNEduKUIVoURXzwwQc4fPgw3n777cx1kvMrJCQEBoMBgwYNwo0bNxAUFIQlS5bkO4g7mr59+zIsExEREaBUQqxbF2LdukDWJTR1Osj++880XF+8CNm1a8USroWEBLj8cxw+/xyHD4DGAHojPVwb69SBvlUd3CldE2eNFZFc/jnEKqvhQbzySc+wgMePBTx8mNFTLENCgnMG7gxGo4AHDwQ8eFCw4zw8MsJ1zj3eGUE8Y7u7O3u9iYqb04VoURQxdOhQ/Pjjj+jZsyfmzJlT4HNIkoSIiIjMf1+8eBH37t1DhQoVrFlqpowe6Jx6mxMTE+Ht7V0s1yYiIiKCUgkxMBBiYKDp9rS0nMO1KFq9DCEhAYpjx6A4dgzVAFR7sl2SySBVrAjRzw+ivz/ERk/+rFIFkr8/dL6V8DhZlRmys4btjKBtKYAbDM6dJhMTBSQmCoiKyv9ISaVSQtmyEnx9Rfj6SihfPvuf6W3lynE5MaLCcqr/dDJ6oDds2IDu3bsXavi1JEkYMWIE1q5di5CQEPzf//0fwsLCEBwcjG3bthVLkM54F/rq1ato3LixSVtsbCySkpLw3HPPWf26RERERLlSqSDWqwcx+9wyWi1kV65kDgeXX7iQHq6vXy+ecC2KEG7dguzWLVhaYFoSBJSqUAEB/v7p4frJl9TgSdiuXBnIttSpJAGJiXgSrGVmAdw0hKcH8ceP00OrM9PpBNy+LeD27dx/RxYECT4+klnQLlfOPHhrNDYqnshJOE2IzhqgQ0JC8N133xV4+LUkSRg5ciRWr16Nbt26YenSpZDL5ZDJZBg8eHDmrN3ly5e3au0tW7bE7NmzsXfvXoSGhpq0/fnnn5n7EBERETkEtRpigwYQGzQw3Z6a+jRcZ8wUnhGuJanYyhEkCUJMDGQxMcDRoxb3EStUSA/UGb3Z/v4o7e8Pb39/iHUrI7/Taut0yCFsy3IM4I8fCw49iZolkiTg/n0B9+8D587l/ju1p6d5sPb1FTN7tTP+9PTkUHIqGZwiRGcM4d6wYQO6du1aqPeXJUnC6NGjsWrVKpMADQDdu6fPMZk1SPv6+lqt/rZt2yIgIACbN2/G4MGDM9eKjo+Px+zZs6FUKgv8XjcRERGRzWk0EBs2hPjkd5lMqamQXb78NFw/mTFcduNGsYbrrGR37kB25w7w998W20VfX9Ne7CxhW/TzQ0Z3q1IJlC+fPuw5v0QRSEiAWdDOadh5xr9TUpwjcSYkCEhIkOPy5dz302gkk55sS0G7fPn097i5Rjc5M6cI0TNmzMD69evh7u6OGjVq4JtvvjHbJygoKDOcWnLnzh1s374dXbt2xdKlS6HI9hJI9+7dIUkShgwZgn379qFXr1651rR69WoceTLcKDIyEgCwZs2azDWhmzdvjv79+wMAFAoFvv32W4SGhiIoKAghISFwd3fH1q1bER0djSlTpqBKlSr5/4EQERERORKNBmKjRhAbNTLdnpJiFq7lFy9CdvOmzUuUxcZCFhsLHD9usV0sW/ZpwM4SrjNDtptbzueWAd7egLe3iGrVctzNTGoqTN7hzqvH+9EjAXFxAiTJMcN3aqqAmzcF3LyZe0JWKNLDtvl72xLKlXsauH19Jbi42Kh4ogJwihAdFRUFAEhKSsLMmTMt7uPv759riK5YsSL27NmDypUrmwXoDD169ECTJk1Qo0aNPGs6cuQI1q9fb7Lt6NGjOJpliFFGiAaANm3aYPfu3Zg+fTq2bNkCvV6PwMBATJ48GSEhIXlej4iIiMjpuLpCbNwYYrY5YZCcDMO5c4j75x+US0mBMiYGsqgoyKKjIYuKgqDV2rxU2f37kN2/D5w4YbFd9PEx78HO2pPt7l7ga2o06b23FSvmv9fbaATi45+G6qezmKcH7gcPZIiNFZ58yXDvnuMNNTcYBMTECIiJybs7ukyZnCZIM50ozdXVBoUTPSHExcXZZowNUR60Wi2io6Ph5+cHdT7fWyKyNd6n5Ax4n5IzyPE+lSQI9++nh+qoKAhPgnXmV3Q0hJQU+xWeA7F0acu92E++4OFhl7qMRuDhQwF376aH6ow/Y2MF3L0rywzbsbECtFrHCtsF4en5dNbxnAO3CC+vgr+3zWcqZecUPdFEREREVEIIAqRy5WAsVw7GF14wb5ckCA8fpgfsLME6a9AWkpNtXrbs0SPIHj0CTp2y2C6WKpVjL7bo7w94eRVLXXI5UK5c+vBpIOeZ1SUJiI9HjkE7a+B2xHW7M97bvnIl9/3Uasli0M54XztjOLmPj4QCTsFEJQhDNBERERE5D0GA5OMDo48PYGmJUEmC8OhReg/2zZtmvdiyqCgISUk2L1v2+DHw+DHkZ85YbJe8vMyCtUlPtrd3sdYnCE/f665dGwCMOe6bkpI1bKcH7Hv3zAP3gweON3uYVisgKirvtbfl8oz1tiWULauGm5sMVatqULGiHL6+YpY2ER4enJW8pGGIJiIiIqJnhyBAKlMGUpky5u9iA+khOy4OQkbAtjRcPCHB9mXHx0N+9izkZ89abJc8Pc3D9ZN/S1WqQPL2tlmSc3UFqlYVUbVq7vvp9cC9e6bDyLMG76zvbRsMjpVCjcb0IfB37wKAHEDZHPfNmJW8XDkxc8K0jJCdsS3jT767/WxgiCYiIiKikkMQIJUqBalUKcshGwDi4iz2YGcOF4+Pt2nJACAkJEB+/jzk589bbJc8PNJDtZ8fJB+f9O/R2zv9K+PvWf/09ERxj1d2cQEqVZJQqVLOvdpA+hJhDx8+nQzN8vvb6X9PTXWssA3kf1ZyAPDwyBqsTYN39m1KpQ2Kp0JhiCYiIiIiysrbG6K3t/l62Bni4817sDN6saOi0odu25iQmAh5ZCTkT5ZezQ/J0zMzWIvZg3bG3728zLbB3d2qvd4yGVC2bPrw6fr1c39vOyEBuQTtp8PK4+MdL2wDQGKigMREOa5ezXtfb2/xSW+2efDOGEru65v+/nYOiw9RMeGPm4iIiIioILy8IHp5Qaxf33J7QkJ6yM7eg53Rq/3woW3rzYGQkJA+dP3mTRSkT1pSKMx7ubN+5RTIvb2BIsxuLQjp8695eYmoVQvI7b3t1FSY9WxbmijtwQPHXXc7Lk6GuDjg0qXc9xMECWXKSDkOJc+6rXRpCTLHe1Xd6TBEExERERFZk6cnxHr1INarZ7k9Kck0YGf0YGd8PXhg23oLSDAYIDx4ABSiTkmjKVDozvzTywsF6W7VaICAAAkBAbkPJTcYTN/btjRBWsZ2vd4x06ckpX8Y8OABEBmZ+8chGROmmYZr817ucuUKtxxYScEQTURERERkS+7uEOvWhVi3ruX25GTIbt2y3IsdFQXZvXu2rdeKhNRUCKmpwJ07BT5W8vQ0HV6eJXSbDUfPsh88PXNMgwoFULGihIoVcw7bWq0WUVHR8PT0R3y8BvfuCbh3Lz1g37+fHsAz/rx3L32b0eiY6dN8wrScKZU5DyV/2sudPqzcyiP8HR5DNBERERGRI3Fzg1i7NsT0tabMpaSkh+yMJbuy9mLHxKTPPm6HtbKLW+bw8+joAh0nyeUW3+222BuerRccGk36tQXA2zt9Xem8lgATReDRI8EkZGcN3hl/v3dPwMOHjjucXKcTcOuWgFu38u6Bd3XNacK0p+9uZ2x78iN1agzRRERERETOxNUVYq1aENNfDLZMp0sP03FxEB4/tvxn1q+sbXq97b4XGxCMRgiPHgGPHhX4WEmlgru3Nzy8vCD384NQuTLE8uUhVqwIqXx5iBUqQKpQAVKZMsh42VgmA3x80if8Spdz4DYYgAcPMgJ3RsjOCN1Ze7kFxMU55nByAEhJEXDjhhw3buS9r6enhJo1jfjzT+f9oIchmoiIiIjoWaNUQipXDlK5cgU7TpKAlJSChe6MP+PjIUhS3tdwIkJaGuSxsXCNjQUuX85xP8nFBZKvr0m4zgjYYvny6X9WqAB4eJgcp1AA5ctLKF9eApDzzOQAkJYG3L9vHrKz9mxn/D0x0TF7twEgIUFAUpLj1pcfDNFERERERJROEAA3N0hubpAqVSrYsaIIJCRkBm2ZpRCeUzBPSiqe78dGBL0ewq1bkN26let+krt7erjO0ottFrjLl4elRaJVKqByZQmVK+c+WRoApKQgW7iWWRxefu+eYJe1t8uVc+4PWxiiiYiIiIio6GQyIOO9YuTVr5qNTpfek53X8POMHu+sbTpdMX1D1ickJUF+5Qpw5Uqu+4k+PpkBOyNcmwTtihVNhpBn5+qav9nJJQlITETmUPKc3uHOaNfrrRO4fX0LdHc4HIZoIiIiIiKyL6USUtmykMqWLdhxkgSkpuY+xDy3NtExw5zsyRJi8rNnc9wncwh51qCd7V1tS0PIsxKE9MnLPT1FVK+ee02SBMTFZQwjz/oO99Ne7Yx3uPOaobxsWfZEExERERER2Z4gAK6ukFxdIVWsWLBjRRFITDQbWi7L1vMt3r8PQ3Q0NI8eQRYbC8GY93BqWyjQEPIs72VbDNw5DCE3uZ4AlColoVQpKc8Zyo3G9BnKs7+vndHL3bSpY/wMC4shmoiIiIiISh6ZDPDySl/+KpfdtFotoqOj4efnB7VSCeHBAwgxMZDdvQvZnTsQ7txJ//Pu3fQlxu7ehezhQ5t9G3kRkpIg/+8/4L//ct1P9PFJD9W5TI4m+fjkOIQ8K7k8vbe5bFkJ9eo5Zm9/UTBEExERERER5YdMljnrea7RMC0tPUxnhGtLQfvOHYdazztzCPm5cznuIykUJr3XOU2OBk9PG1ZuewzRRERERERE1qRSQapSBcYqVXLe58msXlnDtezu3cxe7qwhXDAYbFd7LgSDoXBDyC31bOf2s3FwDNFERERERES29mRWL9HTE6hdO+c3jEUxfQh5tt5s2d27T7fdueNUQ8hFb28k3rhh26KsiCGaiIiIiIjIUWUdQt6oUc77paVBiI3NPWg7yBDyAk8C52AYoomIiIiIiJydSgXJ3x9Gf//c90tIMAnXmQE7a+Au5iHkYvnyxXZuW2CIJiIiIiIiKikyhpDXqpX7EPKHD5++n501cGedHO3Bg0KVIFWoUOjyHQFDNBERERERET0lk0EqWxZS2bK5DyHX6dLDtKWe7SwzkwtJSSaHiQzRREREREREVOIolfkbQp4xC/mTYePG2rVtU18xYYgmIiIiIiKi4uPhAdHDI/ch5E5EZu8CiIiIiIiIiJwFQzQRERERERFRPjFEExEREREREeUTQzQRERERERFRPjFEExEREREREeUTQzQRERERERFRPjFEExEREREREeUTQzQ5FLlcbu8SiPLE+5ScAe9Tcga8T8lZ8F6lrIS4uDjJ3kUQEREREREROQP2RBMRERERERHlE0M0ERERERERUT4xRBMRERERERHlE0M0ERERERERUT4xRBMRERERERHlE0M0ERERERERUT4xRJPdnTx5Ej169IC/vz8qVqyIjh07YsuWLfYuiwgAEBMTg0WLFqFbt26oX78+ypYti1q1aqFfv374559/7F0eUa7mzp0Lb29veHt74/jx4/Yuh8jEtm3b0LVrV1StWhW+vr5o2LAhBg4ciFu3btm7NCJIkoStW7eic+fOqF27NipUqIAXXngBI0eOxI0bN+xdHtmZwt4FUMkWERGB0NBQqNVqhISEwN3dHVu3bsWAAQNw69YtDB8+3N4lUgm3ZMkSzJ07F1WrVsXLL78MHx8fXL16FTt27MCOHTuwbNkyhISE2LtMIjORkZGYPn063NzckJycbO9yiDJJkoRRo0Zh5cqVqFq1KkJDQ+Hu7o47d+7g0KFDiI6ORuXKle1dJpVwEyZMwMKFC1G+fHkEBQXBw8MD586dw6pVq/DTTz/ht99+Q2BgoL3LJDsR4uLiJHsXQSWTwWBA06ZNERMTgz179qBhw4YAgPj4eHTo0AFRUVH4559/4O/vb+dKqSTbunUrSpcujVatWplsP3z4MLp06QI3NzdcunQJKpXKThUSmdPr9ejYsSNcXFxQrVo1bNq0CXv27EHTpk3tXRoRwsPD8emnn+K9997DjBkzIJfLTdoNBgMUCvbzkP3Exsaibt26qFSpEg4ePAgvL6/MtoULF+Kzzz5D3759sXDhQjtWSfbE4dxkNxEREbh+/Tq6d++eGaABwMvLC6NHj4ZOp8P69evtWCER8MYbb5gFaABo0aIFWrdujbi4OERGRtqhMqKczZw5ExcvXsSCBQvMAgqRPaWmpmLGjBkICAjAV199ZfH+ZIAme4uKioIoimjWrJlJgAaA//u//wMAPHjwwB6lkYPgU4rs5uDBgwCA9u3bm7V16NABAHDo0CGb1kRUEC4uLgDAkEIO5dSpU5g1axbGjx+POnXq2LscIhN79+5FXFwc+vbtC6PRiJ07d+Lq1avw8vJCu3btUK1aNXuXSITq1atDqVTi6NGjSEhIgKenZ2bb7t27AQBt27a1V3nkABiiyW6uXr0KIP1BlZ2vry/c3d1x7do1W5dFlC/R0dHYt28fypcvj3r16tm7HCIAQFpaGsLCwtCgQQOMGDHC3uUQmTl16hSA9A8fW7Zsif/++y+zTSaT4YMPPsDUqVPtVB1RutKlS2PixImYMGECXnzxRXTq1CnzneiIiAi89957GDRokL3LJDtiiCa7SUhIAACTT/ey8vDwyNyHyJHo9XoMHjwYaWlpmDRpEnuiyWFMmzYNV69exb59+3hfkkPKGAK7cOFCNGrUCHv37kWtWrVw5swZjBw5EgsWLEDVqlUxcOBAO1dKJd3QoUNRsWJFfPjhh1ixYkXm9ubNm6N79+587aCE4zvRREQFIIoiPvjgAxw+fBhvv/02evfube+SiAAAx44dw/z58zFmzBjOGEsOSxRFAIBSqcS6devw3HPPwd3dHS1atMDKlSshk8mwYMECO1dJBMyYMQODBg3C6NGjcf78edy6dQu7du2CVqtF586dsXPnTnuXSHbEEE12k9EDnVNvc2JiYo691ET2IIoihg4dih9//BE9e/bEnDlz7F0SEYD02YzDwsJQr149jBo1yt7lEOUo4//XGzdujAoVKpi0BQYGIiAgANevX0dcXJwdqiNKt2/fPkyfPh3vv/8+Ro0ahUqVKsHd3R3NmzfHhg0b4OLiggkTJti7TLIjjkMgu8l4F/rq1ato3LixSVtsbCySkpLw3HPP2aEyInMZPdAbNmxA9+7dER4eDpmMn0OSY0hKSsqcZ6Js2bIW93nllVcAAGvXrkXnzp1tVhtRVjVr1gQAsxmPM2Rs12q1NquJKLs9e/YAAFq3bm3W5uvri5o1a+LMmTNISkqCu7u7rcsjB8AQTXbTsmVLzJ49G3v37kVoaKhJ259//pm5D5G9ZQ3QISEh+O677/i+KTkUlUqFfv36WWw7fPgwrl69itdffx0+Pj7w9/e3cXVET2WEksuXL5u16fV6XLt2DW5ubvDx8bF1aUSZdDodgJyXsXr48CFkMlnmKh1U8jBEk920bdsWAQEB2Lx5MwYPHpy5VnR8fDxmz54NpVLJ903J7jKGcG/YsAFdu3bFkiVLGKDJ4Wg0GsyfP99iW1hYGK5evYrRo0ejadOmNq6MyFTVqlXRvn177N27F6tXr0b//v0z2+bMmYP4+Hj07NmTkzaRXTVr1gxLly7FokWL8MYbb5iMnFixYgVu376NZs2aQaVS2bFKsichLi5OsncRVHJFREQgNDQUarUaISEhcHd3x9atWxEdHY0pU6Zg+PDh9i6RSrjp06djxowZcHd3x5AhQywG6KCgoMwPgYgcTVhYGNavX489e/YwRJNDuH79Ol599VXcv38fr732WubQ2IiICPj5+eGPP/6Ar6+vvcukEsxoNCI4OBiHDx9G2bJl8frrr8PLywunT59GREQENBoNtm/fjueff97epZKd8GM+sqs2bdpg9+7dmD59OrZs2QK9Xo/AwEBMnjwZISEh9i7v/9u725iqyz+O428FzIaE3QhsHDUcaJBWyyy11cZmtcFYEzcnOdMHMmRFVJauzXU31rJVDyLLbLOBSS1tSG61VkieJo20zXaKxc00SMpiJVA9kJv4P3CePHLTyfpLyvu1sbHvua7z+/KMz67r+l0S7e3twKkzp88///ywY2bMmGGIlqQopaWlUVdXxzPPPENtbS379u0jOTmZwsJCNmzYMOK5ful8iYmJobq6mldeeYXq6mp2795Nb28vSUlJLF++nPXr1zNnzpyxblNjyJVoSZIkSZKi5KtlJUmSJEmKkiFakiRJkqQoGaIlSZIkSYqSIVqSJEmSpCgZoiVJkiRJipIhWpIkSZKkKBmiJUmSJEmKkiFakiRJkqQoGaIlSZIkSYqSIVqSJF2w2tramDp1KlOnTh3rViRJ40TsWDcgSZL+v3Jzczlw4EBUY7u6uv6/zUiSdIEzREuSNE4EAgECgcBYtyFJ0gXNEC1J0jixcuVKHnvssbFuQ5KkC5pnoiVJkiRJipIhWpIkDXH2C7s++OADcnNzmTlzJqmpqSxZsoR33nln1O9oamrivvvuY968eSQlJTFz5kxycnKorKxkYGBgxHn9/f1UVVWRn59Peno6SUlJZGZmkpOTQ3l5Od3d3SPO/eyzz1i+fDlpaWmkpKSwePFitm3bxuDg4LDjDx8+TGFhIXPnziUpKYnU1FTmzZvHsmXLKC8vH3GeJGn8cju3JEka1WuvvcbGjRu5/PLLmTVrFh0dHRw6dCj889xzzw2ZU11dTVFREb29vcTHx5OVlcWJEyeor6+nvr6e6upqqqqquPTSSyPm/fzzz9xzzz00NDQAkJyczNy5c+ns7KShoYH6+npuuOEGbrvttiHP3LlzJyUlJSQmJnL11Vfz3Xff0djYyIYNG2hvb6esrCxi/Mcff0xBQQF9fX1MmTKF9PR0YmNj+f7776mtraW2tpbi4mJiY/13SZL0J1eiJUnSqDZt2sSjjz5KS0sLdXV1NDU18eKLLzJx4kS2bdvGnj17IsY3NzdTXFxMb28v9957L83NzXzyySd8+eWX7Nmzh8suu4y6ujoef/zxiHmDg4OsXr2ahoYGAoEANTU1NDU1sW/fPkKhEEeOHOGFF14gKSlp2D4ffvhhysrKaG1tpa6ujtbW1vAztmzZwtGjRyPGP/nkk/T19VFaWkpLSwv19fUEg0FaW1sJhUI89dRTTJzov0qSpEgTurq63KckSdJFLNorrnJycqiqqgJObee+/vrrAcjOzqa6unrI+NLSUioqKsjKyqK+vj5cv//++3nzzTfJysriwIEDTJgwIWJeZWUlDzzwAHFxcYRCIVJSUoBTW8YLCgq45JJLCAaDzJkz5y97PrPPVatWUV5ePmTM4sWLaWxs5Nlnn2XdunXhenJyMidPnqStrY3ExMS/fJYkSeBKtCRJ40YgEGDhwoUj/lxzzTXDzisuLh613tjYyLFjx8L1jz76CIB169YNCdAABQUFTJs2jb6+Purq6sL19957D4C8vLyoAvTZ1q5dO2z95ptvBuDIkSMR9enTpwOwe/fuv/0sSdL45SEfSZLGiXO94iozM3PYekZGBrGxsfT399Pc3EwgEKC7u5sff/wRgKysrGHnxcXFkZGRQWdnJy0tLeF6Y2Mj8Gfo/bvS09OHrU+bNg2A3377LaJeWlpKSUkJ69ev5+WXXyY7O5sFCxZw6623MmPGjHPqQZJ08XMlWpIkjWqkM8gxMTFcccUVAPz6669AZFAdaR4Q3sJ9et6Zv5/r1ur4+Phh66fPNZ/9pu1Vq1axY8cObrnlFtra2ti+fTvFxcVcd911LFmyhE8//fSc+pAkXdwM0ZIkaVQ//fTTsPWBgQF++eUXABISEgCYMmXKX84DOH78eMS8M38f7Qqrf1teXh4ffvghR48eZdeuXTz00EOkpaVx6NAhli1bRigUOm+9SJIuDIZoSZI0qm+++WbYektLC/39/QDMnj0bOLWKnJycDPy5Pfts/f394W3cp+cBXHvttQB8/vnn/07jf0NiYiJ33HEHTzzxBAcPHmTBggX09vZSWVl53nuRJP23GaIlSdKotm7dOmo9KyuLQCAQrt95553hz8/eQg3w9ttv09nZSVxcHNnZ2eH63XffDcDevXsjzkqfb7GxscyfPx+AH374Ycz6kCT9NxmiJUnSqILBIJs3bw6vOg8ODlJRUcGOHTsAeOSRRyLGl5SUMHnyZBobG3nwwQf5/fffw5/t37+fTZs2AbBmzZrwqjXAXXfdxe23387JkyfJz88fcia5p6eH7du309TU9I//pp6eHlavXk1tbS29vb0Rnx0+fDh8pdeNN974j58lSbq4+HZuSZLGiZ07d7J///5Rx2zevDl87/JpZWVlbNy4ka1bt5KWlkZHR0f4Ddxr164lPz8/Yvzs2bN59dVXKSoqoqKignfffZeMjAxOnDjBt99+C5y6e/rpp58e8vw33niDFStWcPDgQfLy8khJSSE1NZXOzk46OjoYGBhg796953QF1pn++OMPampqqKmpYdKkScyaNYv4+Hg6Oztpb28H4Kabboq4V1qSJDBES5I0bhw7diziPufh9PT0DKkVFRUxffp0tmzZQigUor+/n/nz51NYWMiKFSuG/Z6lS5eSmZnJSy+9RDAY5Ouvv2by5MksWrSIgoICVq5cSUxMzJB5V155Je+//z5vvfUWu3bt4quvviIUCnHVVVexcOFCcnNzh4T8c5GQkMDrr79OMBjkiy++4Pjx43R3d5OQkMCiRYtYunQpa9asYdKkSf/4WZKki8uErq6uoYeVJEnSuNbW1hYOq11dXWPbjCRJ/yGeiZYkSZIkKUqGaEmSJEmSomSIliRJkiQpSoZoSZIkSZKi5IvFJEmSJEmKkivRkiRJkiRFyRAtSZIkSVKUDNGSJEmSJEXJEC1JkiRJUpQM0ZIkSZIkRckQLUmSJElSlAzRkiRJkiRFyRAtSZIkSVKUDNGSJEmSJEXpf0JAz6RxgfHSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_transf.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HPQvdYWVPIpY",
    "outputId": "7bc90613-c4cc-4f84-fb78-741a6f061376"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[430, 447],\n",
       "        [284, 324]])"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_transf.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8Xh8xxgTPIpY"
   },
   "source": [
    "### Visualizing Attention"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "i5_67okaPIpY",
    "outputId": "6aad22a7-dce6-4973-f9dd-e697587044fd"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[    2,   300, 12427,     7,  7101,  1423,   422],\n",
       "        [    2,  6659,   811, 11238,    15,     2,   588]], device='cuda:0')"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentences = ['The white rabbit and Alice ran away', 'The lion met Dorothy on the road']\n",
    "inputs = glove_tokenizer(sentences, add_special_tokens=False, return_tensors='pt')['input_ids']\n",
    "inputs = inputs.to(sbs_transf.device)\n",
    "inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Cu3MXRxgPIpZ",
    "outputId": "d75679aa-df51-44f6-a5a6-55cd7e1325f6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.9883],\n",
       "        [0.0093]], device='cuda:0', grad_fn=<SigmoidBackward0>)"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sbs_transf.model.eval()\n",
    "out = sbs_transf.model(inputs)\n",
    "# our model outputs logits, so we turn them into probs\n",
    "torch.sigmoid(out)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4Nld9xIhPIpZ",
    "outputId": "7915002f-398d-4011-e276-b08d3b5090af"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[2.9222e-01, 6.1641e-02, 1.7531e-01, 1.4711e-01, 9.3683e-02,\n",
       "          1.6146e-01, 5.5209e-02, 1.3368e-02],\n",
       "         [5.5408e-05, 4.6049e-03, 4.9222e-03, 1.3957e-01, 3.1658e-03,\n",
       "          8.4394e-01, 1.7378e-03, 2.0026e-03]],\n",
       "\n",
       "        [[7.2964e-02, 1.5391e-02, 1.0228e-01, 6.3236e-02, 6.4430e-01,\n",
       "          9.5831e-03, 1.4200e-02, 7.8049e-02],\n",
       "         [3.7525e-04, 3.1186e-02, 3.3432e-01, 3.3407e-02, 4.6640e-01,\n",
       "          5.5731e-02, 3.7347e-02, 4.1234e-02]]], device='cuda:0')"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alphas = sbs_transf.model.encoder.layers[0].self_attn_heads.alphas\n",
    "alphas[:, :, 0, :].squeeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "id": "B5gATXCkPIpZ",
    "outputId": "c0e7ea1e-a714-4172-e335-fe313a82d076"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/content/plots/chapter11.py:50: UserWarning: FixedFormatter should only be used together with FixedLocator\n",
      "  ax.set_yticklabels(['', f'Head #{i}', ''])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABbIAAADNCAYAAABtqYY+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACN9UlEQVR4nOzdd1AUSRsG8AcRMYAERQwICBjOU1HEiGAAEyCCCfOZc/ZQMWfMnjmdOSKKEVRMICBgzlkEc0JRQAEJ3x/UzMe6CyywwOo9vyqqztnZ3p6+np6ed3q6VaKjo1NBRERERERERERERKSkChV0BoiIiIiIiIiIiIiIMsNANhEREREREREREREpNQayiYiIiIiIiIiIiEipMZBNREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZBMRERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZRERERERERERERKTUGMgmIiIiIiIiIiIiIqXGQPZvYPfu3dDW1hb/+vXrV9BZypSdnZ1EfgMDAws6S3IbOnQotLW1ERkZWdBZUTqBgYHQ1taGh4eH3N+pWbMmatasmYe5Uk75UVZCu7B79+6cZPG3pq2tDQcHh4LORq5kVIccHBygra1dMJkiIiKFYN8+/7BvD0RGRkJbWxtDhw4Vt7FcSFa9UGYeHh6/XPtDucN2Snnk9/nHQPZvxN7eHhMnTkT79u2lPvv27RvWrVsHR0dHmJqaonTp0jA2NkabNm2wfPlyfPz4UWJ/bW1t1KtXT67fjYuLw9KlS2FjY4MKFSqgTJkyqF69Otq2bYtZs2bh2bNnEvv37t0bEydOhJWVVc4PNg/lJMhIisNOSN76HYK4RERE/wXs2ysG+/b0K+KgFEm/WmCdFIPtN8lSuKAzQIrj4OCAHj16SG2/ffs2unfvjhcvXqBixYpo27YtypQpg5iYGFy+fBmzZs3C8uXLcf/+fZQoUSJbvxkTE4M2bdrg7t27MDExQZcuXaCrq4uoqChcvXoVy5cvR6VKlVCpUiXxO7179waQFrAMDg7O3UHTL+3o0aMFnYXflqOjI+rVqwd9ff2Czgrlo/Xr1+P79+8FnQ0iIlIA9u2poMyYMQNjx45F+fLlCzorREREEhjI/s29evUKHTp0QFRUFObOnYuhQ4dCVVVVYp+bN29iwoQJ+PHjR7bTX7duHe7evYvevXtjxYoVUFFRkfg8IiICiYmJuToG+n2lvwkixdLS0oKWllZBZ4PyWcWKFQs6C0RElIfYt6f8ULZsWZQtW7ags0FERCSFU4v85ubMmYMPHz5g3LhxGDFihFRHFwDMzc3h4+ODkiVLZjv9y5cvAwAGDBgg1dEFAGNjY1SpUiX7GS8gHh4eaNeuHQBg4cKFEvP9pZ97KTU1FevXr0e9evVQpkwZ1KhRAwsWLEBKSorMdH18fODk5AQjIyPo6+ujUaNGWLVqFZKTk/PluOQRHR0NXV1duLq6Smy/deuWWAbh4eESnzk4OKBs2bJISEiQ2H79+nU4OzvDwMAAhoaG6NGjh8y5q36e99nBwQELFy4EALRr10783Z/nhv7w4QPc3d1Rp04dlClTBiYmJujVqxfu3buXqzLITPrXmsLCwuDi4gJDQ0NxPuKdO3eiW7duqFmzJvT19WFsbIwOHTrgwoULmaYbEhICBwcHsax69eolVc7pRUdHY8yYMahSpQr09fVhbW2NAwcOSO338+uIQv4BIDg4WKJuF8Qri4mJidiwYQM6dOiAP//8E2XKlIGZmRl69uyJmzdvSuyb/ljOnTuHVq1aoVy5cqhUqRKGDBmCT58+yfyNHTt2oFGjRtDX18eff/6J6dOnIz4+Pj8OL0eyUyYZyWyObB8fH7i4uKBSpUrQ19dHzZo1MWjQIKnzJjExEatXr4aNjQ3Kly8PAwMDtG3bFr6+vrk9xDynyPM0fVrytmkFSd76c/v2bWhra8PNzU3i+8ePH4e2tjbKlCmDb9++SXxWs2ZN1KpVS/z3ly9f8M8//8De3h7VqlWDnp4eqlWrhsGDB0tNOTB37lxoa2vj0KFDMvO9c+dOaGtrY9myZbktAqL/BPbts+e/3LfPjczmnt21axdsbW1RoUIFVKhQAba2tjL7kr/adTQjP/ctHB0dYWBgAFNTU4wfP158E+7UqVNo2bIlypcvj8qVK2P69OlISkqSSk+eujN06FAMHz4cADB8+HCJeqtoycnJ+Oeff1CnTh3o6+ujTp06WLZsGVJTU2Xuf+/ePfTp0wdmZmYoU6YMatWqhUmTJsnsjwv3etHR0XBzc8Off/6JUqVKSdSXEydOwNHREYaGhihbtiysrKywevVqibLbvXs3zM3NAQB79+7Nck5+Ly8vNGnSBGXLlkXVqlUxceJEiTcW/f39oa2tjfHjx8s8xmfPnkFHRwcdOnSQrxAVSJ7z63c5t7LC9ltaVvc6cXFxmD9/vvhmtrGxMbp06YLQ0FCptN68eYP58+fDzs5OPJ9r1qyJ8ePH48OHDzJ//+XLl+jfvz+MjY1RoUIF2NvbF8ibWByR/Rv79u0bvL29UaxYMYwcOTLTfQsXzllV0NHRAQA8ffpU4ib3V9WkSRM8f/4ce/fuhZWVFZo0aSJ+ln506/Tp0xEcHIzWrVujRYsW8PHxwYIFC/Djxw9MmzZNIk3h9c7y5cujXbt2KFmyJEJCQjBt2jRcuXIF27dvz7fjy4y2tjZq1KiBkJAQJCcnizdG6TsHgYGBMDExAQDEx8fjypUrqF+/PtTV1cV9rl+/jpUrV8La2hp9+vTBrVu34OPjg3v37iEkJARFixbNMA/du3cHkBZo7datGwwNDQFIlv2zZ8/g6OiIV69eoUWLFnBwcMCHDx9w7NgxnDt3DkeOHIGlpaXiCuYnly5dwrJly8Tje/nyJQDAzc0NNWrUQLNmzVC6dGm8fv0avr6+cHZ2xs6dO2XOS33lyhUsX74ctra2GDRoEO7fv4/jx48jJCQEZ86cgbGxscT+P378gLOzM+Li4uDq6opv377h0KFDGDBgAKKiojB48OAM821oaIiJEydi4cKFqFixoljWAApkwc3Pnz/D3d0djRo1QsuWLaGtrY2IiAicOHECZ86cga+vLywsLCS+c+LECfj5+aFNmzaoX78+Ll68iH379iEiIgInT56U2HfRokWYP38+ypQpg969e0NNTQ3e3t54+PBhfh5mtuSkTOQ1ZcoUrFmzBjo6OnBwcICenh5evXqFgIAA1K5dG9WrVwcAJCQkoGPHjggKCkLNmjXRs2dPJCUlwc/PD927d8eiRYswaNAgRR52nlDkeZqbNi0/yVt/atSoAV1dXakbP+HfiYmJCAsLQ/PmzQGkjb588eKFxPQGjx49wvz582FtbQ1HR0cUL14cjx49woEDB+Dn54eAgACx/e7duzeWLVuGHTt2wMXFRSrfO3bsQOHChWVOn0BEkti3z77/ct8+L0yYMAEbN25E+fLl0bNnTwDAsWPHMHz4cNy6dUsckJLer3IdzcrVq1exYsUKtGjRAn369EFgYCA2b94sTsczbNgw2Nvbo169evDz88PKlStRokQJTJw4UUxD3rrj4OCAL1++wNfXF/b29nnaVx89ejR27doFIyMjDBgwAAkJCVizZg3CwsKk9g0JCUHHjh2RmJiI9u3bw9DQEJcuXcL69etx6tQpnDlzBqVKlZL4TmJiIpycnBAXF4e2bdtCVVUVZcqUAQCsXr0aU6dOhY6ODjp16oTixYvjxIkTmDp1KkJCQrBr1y6oqKigZs2aGDJkCNavX48aNWpI9NeE/oZg06ZNOHv2LOzt7WFjY4OzZ89iw4YN+PTpEzZt2gQAaNq0KSpVqgQvLy/MmTMHxYsXl0hjx44dSE1NxV9//aWQMpZXds+v3+Xcygjb74zJuteJj4+Hk5MTrl69CnNzcwwdOhTv37/HoUOHcPbsWWzevBnOzs5iGhcvXsSaNWtgY2ODunXrQk1NDbdu3cLmzZtx9uxZBAQESJTz27dv0apVK7x+/Rq2trYwNzfHw4cP4eLiAmtr63w9fgayf2PXrl1DYmIiGjVqlGdTDDg7O2P//v0YNWoUrl69ihYtWqB27drQ1dXNk9/La8IJuHfvXjRp0gTu7u4y97t58yaCg4PFV+4mTJgACwsLbNy4ERMnTkSRIkUAAOfPnxcDlTt27BDnKUxNTcX48eOxZcsWHDlyROYiPgXB2toat27dwo0bN1C3bl0AacENMzMzxMfHIzAwULygh4WFISEhQarR8vPzw5YtWySeYA8ePBienp7w8fFBx44dM/z9Hj164Pnz5wgODkb37t1lNohDhgzB27dvcfDgQdja2orb3dzc0Lx5c4waNQoXL17MVTlk5vz581i9erXYuRCEhoZKBZ7fvn2L5s2bY/r06TIDZGfPnsXy5cvRt29fcdvWrVsxduxYTJw4EZ6enlLpmZiYwM/PT6xj48aNg42NDaZPn4527dplOJehkZER3N3dsXDhQhgaGmZYt/OLtrY27ty5I5Xf+/fvo2XLlpg9ezYOHz4s8dnJkydx/PhxNGzYEEDaCJL27dsjKCgIly9fFhexCg8Px6JFi1C+fHkEBARAT08PADBp0iSJOqNsclIm8jh58iTWrFmD6tWr4/jx4xLtc1JSksQImkWLFiEoKAhubm6YPHmyOBovJiYGTk5OmDp1Ktq1a4dy5crl7CDziSLP09y0aflJ3vqjoqICKysrHDt2DO/fvxdvJgMDA9GoUSNcvXoVgYGBYiBbCHCnb4+rVKmChw8figEvwYULF+Ds7IwlS5Zg5cqVANJuMG1tbXHmzBlERkbCyMhIIm+XL1+Gg4MD5/MnkgP79tn3X+/bK1JwcDA2btyIqlWrws/PT6yD7u7usLOzw4YNG9C+fXs0btxY4nu/ynU0K2fOnMHu3bvFvsKPHz/QrFkzeHl54ezZsxIDDtzd3WFhYYH169dj3LhxUFNTy1bdcXR0FAPZGc2VrwiBgYHYtWsXatSogVOnTol5GjdunNR9WEpKCoYNG4Zv375J3YdNnz4dK1euxIwZM7B69WqJ7717905Mv1ixYuL2Z8+eYebMmdDT08P58+dhYGAAAJg2bRqcnZ3h4+MDT09PdO3aFbVq1YKWlhbWr1+PmjVrZnof4+/vD39/f1SuXBkA8P37d1hbW+PgwYOYPXs2ypUrBxUVFfTp0wczZszA4cOHJQb4JCUlYe/evdDT04O9vX0OSzb7cnJ+/S7nVkbYfmdM1r3OwoULcfXqVXTp0gUbNmwQ7+MGDx6Mli1bYvTo0bC1tYWmpiYAwMbGBg8fPoSGhoZE2nv37sXQoUOxadMm/P333+L2WbNm4fXr15g6darE9m3btmHMmDF5eLTSOLXIb+z9+/cAkKeLdNjb22Pu3LlITU3F6tWr0aFDB5iYmKBOnTpwc3PD06dP8+y3C5Kbm5vEvHGlSpWCvb09YmJi8PjxY3H7xo0bAQD//POPxGI7KioqmDFjBlRUVHDw4MH8y3gWhIuF8Jp9cnIyLl68CGtrazRp0kRqdDYAiSejANC4cWOp17CEBvbatWu5yt/NmzcRFhaGbt26SQUkzczM0Lt3b9y7dy9PpxgxNzeXCo4BkAqOAWnzC7Zr1w5Pnz7F8+fPpT43MzOTetL/119/wdTUFH5+fvj48aPUd6ZPny5ejAGgQoUKGDJkCBISEpSqLmVFXV1dZtv0xx9/oEmTJrh48aLU3J6dOnUSg9gAoKqqim7dugGQrFteXl5ISkrCsGHDxCA2AJQsWVLioqtsclIm8ti8eTMAYMGCBVKBiMKFC4uBzJSUFGzevBmVKlWSCGIDgKamJiZMmIDExEQcO3Ys23nIb4o8T/OyTVOk7NQfoa0X2vGoqCjcv38frVq1gqWlpcRUK7Laei0tLakgNpDWIa5WrRr8/f0ltvft2xepqanYuXOnxPYdO3YAQL6PeCL6VbFvn3d+1769Iu3duxdA2sCA9A9StLW1xVHHe/bskfrer3IdzYq1tbXEA281NTW0b98eqampaNOmjcRbc5qammjdujU+f/6MV69eAVDOurNv3z4AaYG/9HkqX748hgwZIrFvaGgonj17hpYtW0rdh02YMAE6Ojo4cOCAzDn0Z82aJRHEBv7fXx8xYoQYxAbS+jMzZ84EILs+ZWXIkCFiEBsAihUrho4dOyIlJQU3btwQt/fo0QNFihSR6pucOnUKb9++Rbdu3aCmppbt38+pnJxfv8u5lVv/xfZb1r3O3r17oaamJh5L+n27deuGL1++wMfHR9yup6cnFcQGgK5du6JkyZIS/fnExEQcOnQIenp6GDFihMT+vXv3hqmpqYKOTD4ckU25NmLECPz11184e/YswsLCcOPGDVy5cgWbNm3Czp07sWXLlnx9mpkfateuLbWtQoUKANLmDhVcuXIFJUqUwK5du2SmU6xYMYnGtaA1btwYqqqqCAwMxNixY3Hr1i18/foVNjY2+PbtG/bt24eHDx+iatWqCAoKQrFixaSm8ZC3bHLiypUrANLmyPbw8JD6XCjLx48fi1MlKFpGUztERERg2bJluHDhAt68eSM1b/jbt2+lXn1r0KABChWSfJ5YqFAhNGjQAE+fPsWdO3fQrFkz8bPChQujfv36Ur/dqFEjAGlz3/5Kbt26hZUrVyI0NBTv3r2TCtJGRUVJdErkrVt37twBAKkRQcD/y0pZZbdM5HH16lWoq6tLPXT62ePHjxEdHY1y5cphwYIFUp9HRUWJ+yk7RZ6nedmmKZq89Sd9IFuYSiY1NRU2NjaIj4/H0qVLERMTA01NTQQFBaFSpUoSN5nCd9etW4erV68iKipKYi7L9A/bAKB169YoX7489uzZA3d3d6iqqiIxMRGenp4wMDCAnZ1dHpUIEeUE+/Zpfoe+vSLdunULgPQgFuD/D0hl9UV/petoZmRN7yH0yTL77O3btzA2NlbKupOdPnNm//81NDRQp04dnDt3Do8fP8aff/4pfla0aFGJf8uTXv369VG0aNEc3dvIW99Kly6Ndu3a4eDBg3j06JE497/wkL13797Z/u3cyMn59bucW7n1X2y/f77X+fr1KyIiIlC1alXx2NOztrbG9u3bcfv2bXTt2lXcfvToUWzbtg03b95EdHS0xDzhb9++Ff/78ePHiI+Ph42NjdSUNenjF/mFgezfmDDK7vXr13n+W5qamnB2dhbn3Pny5QvmzJmDf//9FyNHjoSdnZ3Uje2vTHgdIz1hTun0J//nz5+RlJQkc744QVxcnOIzmEMlS5aEubk5wsLC8OPHDwQGBkJFRQXW1tbi4l+BgYGoWLEirl69CisrK6n/r/KWTU58/vwZQNqT8lOnTmW4X16WafoRvoLw8HC0aNECMTExsLa2Rps2baCpqYlChQohKCgIwcHBUgEz4P/naEbbf+6AlCpVSirwndn+yiwsLAxOTk4AgObNm6N9+/YoUaIEVFRU4OPjgzt37kiVmbx16+vXrwDSOqg/y6jMlUFOykQeX79+Rbly5WTWnfSE8+v+/fu4f/9+hvspU5uVEUWep3nZpilSdurPH3/8AT09PXG0dWBgIEqWLInatWvj+/fvWLhwIUJCQmBqaorXr19L3cwdPnwYffv2hYaGBlq0aAFDQ0MUK1YMKioq2LNnD168eCGxv6qqKnr16oWFCxfi9OnTaNOmDY4fP45Pnz5h4MCBWdZNIkrDvn3e+V379ooUExODQoUKZdi/UlFRQUxMjNRnv8p1NCuZHUdmnwkPlZWx7nz9+hWFChWSmtcakO4zC/9vZfWxAIhThP1cB0qXLi1z4djM0lNRUYGenh7evHkjx1FIyk5969OnDw4ePIgdO3Zg7ty5ePPmDc6cOQMrKyuYmZll+7dzIyfn1+9ybuXWf7H9/vm8ycn5uWrVKkybNg2lS5dGixYtUL58eTFIvW7dOon7oszur4H8v8dmIPs3ZmFhgSJFiuDGjRv4+vVrjlYuzyktLS0sXrwYp06dwosXL3Dv3j2ZT8p+d5qamlBRUUF4eHhBZ0Vu1tbWuHbtGq5evYqgoCD88ccfYoNlZGSEwMBAmJqa4sePH/k+qb9wkSrIBedkdcTWrl2L6OhobNiwAa6urhKfjR07NsOVfIVXhDPa/vP8l1FRUUhJSZEK+mS0vzJbunQpEhIScOLECakRH1euXBFHiOSE0NZ9/PhRanRtRmWuDPKqTLS0tPD+/XuZdSc94fxycnISR6P8qhR5nv4qslt/mjRpgkOHDuH169cICgpCo0aNoKqqinr16qFYsWIIDAwUbyB/busXLFiAokWLwt/fX+pVQm9vb5n56927N5YsWYLt27ejTZs22LFjBwoVKiRzChgiko19+4L3K/btFUVTUxMpKSn4+PGjVLDkw4cPSE1NlRlQojTKWHdKliyJlJQUREVFSQWofu4zC/9vP3z4IDMtYf+f64CsPtnP6f3cX09NTcWHDx/yvD5ZW1ujSpUq2LdvH6ZPn47du3cjOTm5QKY84/mV95TxHMypn8+r7J6fSUlJWLx4McqWLYvAwECJOpeamiqudSNIf3+dWfr5hUNgfmPFixdHhw4d8P37d6lFF36WlJSElJQUhf6+ioqKxNxDvwpFPsW0tLTEp0+ffqn5BIWAxblz5xASEiIRwLCxsUFQUJA4f2pWUxXkhFD+suqjMI3J5cuXFf67ufHs2TMAkHrNNjU1VeaK34KwsDCp40xJScGlS5egoqKCGjVqSHyWlJSES5cuSaUTEhICQPZrjT8rVKiQws/1nHj27Bl0dHSkAm7fvn3DzZs3c5W2UG6yFv0UykoZ5VWZ1K1bFwkJCQgKCsp0v6pVq6JkyZK4fv16jubiVnY5PU9/FdmtP0LbfvDgQTx48AA2NjYA0uamrF+/Pi5cuJDhWgjPnj1DlSpVpILYb9++RUREhMz8VahQAa1atcLp06cRFhaGgIAA2NraomLFijk6XqL/Ivbtc+a/3rdXlFq1agGAzP6EsE2evuh/VXbrTn6MrM1Onzmz//9xcXG4fv06ihUrJjE/dWYyS+/KlSuIj4+XqE95VR5//fUXPn78CB8fH+zatQva2triG275ieeXbGy/5VOyZEkYGxsjPDxc5ltbP9ehqKgofP36FfXq1ZN6cHL9+nV8//5dYpuZmRmKFi2K69evIz4+XuIzIX6RnxjI/s0JrwosXboU69evl9mhvXPnDhwdHcXXBbJj69atGS4kcPz4cTx8+BBaWlr4448/sp12QREWsBIW5siNwYMHA0iba/DTp09Sn7979w4PHz7M9e8oUsOGDVG4cGFs2bIFMTExYnADSAt8REVFYefOnShRokSG89DmhlD+L1++lPqsbt26sLS0xIEDB2SO+ktJSckyWJcXhEBMaGioxPbly5dnuvDkkydPsH37dolt27dvx5MnT9CqVSuZr+7Mnj1bYhGVV69eYf369VBXV5drZWodHR2F1O3cqlixIqKjoyWmsEhOTsa0adMyfNIrr86dO0NVVRVr166VeCr99etXLFmyJFdp56W8KpMBAwYASFs8Rpg+RJCUlCQ+QS9cuDD69euHFy9eYOrUqTKD2ffu3cvwSb+yy+l5+qvIbv0RAtkrVqwAAKm2/vbt2zh37hzMzMxQrlw5qd969uyZxOiL+Ph4jBs3LtOHIH379kVSUhL69OmD1NTUfJ9/kuh3wL599v3X+/aKIiywvXDhQom69eXLF/FVfWEfkpbduqPIepsR4Q21RYsWSUyp8Pr1a6xfv15i34YNG6JSpUo4ffq01KLOS5YswadPn9CxY0e5pxzq3LkzChcujDVr1khMIZKYmIgZM2YAALp37y5u19bWhoqKisLLo3v37ihatCgmT56MiIgIuLq6Ss0BnB94fsnG9lt+3bp1w48fPzBr1iykpqaK2+/cuYM9e/agZMmS4oK1enp6KFasGG7evClOIQsA0dHRmDBhglTa6urqcHZ2xocPH6QepO/YsQNPnjzJo6OSjVOL/OYqVKgAb29v9OjRA5MmTcLatWvRtGlTlClTBl+/fsW1a9dw7do1aGpqSq3K++7dOwwdOlRmuqVKlcLcuXNx+vRpjB07FiYmJmjQoAHKlSuHuLg43Lp1CyEhIShUqBCWLl0KdXX1/DhchahSpQrKlSsHb29vqKuro3z58lBRUcnRVBZ2dnZwc3PD4sWLUadOHdjZ2aFixYr49OkTwsPDERISgqlTp6Jq1ap5cCQ5o6GhAQsLC1y6dAmFChWClZWV+JkQ+Pj48SNsbW3zZCVna2trqKioYM6cOXjw4AFKliwJLS0tsfz//fdftGvXDv369cO6detgbm6OokWL4uXLl7h8+TI+fvyId+/eKTxfmenbty92796N3r17w9nZGbq6urhy5Qpu3ryJ1q1bZzift62tLSZOnAg/Pz/88ccfuH//Pk6ePIlSpUrJnLurbNmy+PbtG6ysrNCmTRt8+/YNhw4dwqdPn7Bw4UKUL18+y7za2Njg0KFD6N69O2rVqgVVVVW0bdtWavR3Xhs0aBDOnTuHNm3awMXFBerq6ggKCsKbN2/QpEmTXD2QMDExwYQJE+Dh4QErKys4OzujcOHCOHr0KP7880+lXcQjr8qkVatWGDlyJFatWgULCws4OjpCT08Pr1+/xoULFzBixAgMGzYMAODu7o6bN29iw4YN8PPzQ+PGjcV97927hzt37uD06dMZzr+mzHJ6nv4qslt/KleujLJly+Lt27fQ1dWVaAOsra2RkpKCT58+oX379jJ/a8KECbCxsYGTkxOSk5Nx/vx5pKamokaNGhlOgyNcA1+8eAF9fX20bdtWsYVA9B/Avn32/df79opiZWWFQYMGYePGjWjcuDHatWuH1NRUHDt2DK9evcLgwYMl7htIUnbrTv369VGsWDGsW7cO0dHR4gAXNzc3heXJxsYGPXr0wO7du9G4cWM4OjoiMTER3t7esLS0lOgbFSpUCGvXrkXHjh3RuXNnODs7o2LFirh06ZK4MPTMmTPl/m1h/6lTp8LKygouLi4oXrw4Tp48icePH8Pe3l5iKjjhHvXixYsYNGgQTE1NUahQIbi6ukpNTZIdOjo6aN++PTw9PQGgQKYVAXh+ZYTtt/xGjx4NPz8/eHp64tGjR2jatCk+fPiAQ4cOISkpCRs2bBCnFilUqBD69++P1atXo0mTJmjTpg1iYmJw5swZVKxYUWoQCwDMnDkTFy5cwNy5cxEaGopatWrh4cOHOH36NFq0aIFz587l27EykP0fUKtWLYSFhWH79u04fvw4fH198eXLF5QoUQJVq1bF1KlT0bdvX6lXBb9+/Yq9e/fKTLNixYqYO3cuZs2ahYYNG+L8+fO4ePGiGEAsV64cunXrhsGDB/9y8+epqqpi586dmDFjBg4ePChOiN+lS5ccpTdlyhRYWVlh/fr1CAgIwJcvX6CrqwsjIyNMmjQJnTt3VmT2FcLa2hqXLl1CrVq1oK2tLW4vV64czMzM8OTJkzyZVgQAqlWrhjVr1mD16tXYuHEjEhISULFiRfFiZWxsjMDAQKxevRq+vr7YvXs3VFVVoa+vj8aNGxfIq2Dm5ubw9vbGvHnzcPz4cXHl3pMnT+LEiRMZBsgsLS3x999/Y+7cudiwYQNUVVXh4OCA2bNnw9jYWGp/NTU1HD58GDNnzoSnpye+fPmCypUrY9GiRejUqZNceV2wYAEA4MKFCzh58iRSUlJQvnz5fA9kt2nTBtu3b8eyZcuwf/9+FCtWDDY2Nti9e3emC3DIa+LEiShXrhzWrl2Lbdu2QU9PDx06dMDkyZNlXpiVQV6WyZw5c1CvXj1s2rQJR44cQUJCAvT19WFtbY3mzZuL+6mrq+PAgQPYuXMn9u3bh2PHjiEhIQF6enqoVq0a+vXrh+rVq+f2UAtETs/TX0VO6o+1tTW8vLzQpEkTibn2LCwsoKGhgdjYWJlt/cCBA6GmpoaNGzdix44d0NLSQqtWrTBjxoxMbwCFG84lS5age/fuKFyY3VCinGDfPnvYt1ecRYsWoVatWtiyZYv4VmG1atXg7u7ONQ/kkJ26o6Ojg+3bt2PBggXYsWOH+Kq/IgPZALBy5UqYmZlh+/bt2LRpE8qXL4/hw4fDxcVFqm/UqFEjnD59GosWLcK5c+fw9etXlC1bFkOGDIGbm5vMRSMzM2LECJiYmGDNmjXYv38/EhMTYWpqirlz52LIkCFS8wBv2LABkydPxqlTp/D161ekpqaiYcOGuQpkA2mjsj09PVGvXr0C7efy/JLG9lt+RYsWxdGjR/HPP//g0KFDWLt2LYoVKwYrKyuMGzdOavrBGTNmQEdHB3v27MHmzZuhp6eHjh07YtKkSVL7AmmD6k6dOoUZM2bg7NmzuHjxIszNzXHo0CFcuHAhXwPZKtHR0alZ70bKbPfu3Rg+fDjWrFmDHj16FHR25Obh4YGFCxfi2LFj+b5oIBEREeU/V1dX+Pn54erVqzAxMSno7BApJfbtiYjyz6pVqzBt2jSsWrUKvXr1KujsEFEWOEf2b2T48OHQ1tZGv379CjormbKzs4O2trZCRl0SERHRr+HBgwfw8/ND8+bNGcQmkgP79kREeSs+Ph4bN26Etra2XOsNEVHB4zudv4GaNWti4sSJ4r+V/bXv3r17w9bWVvx3bl8FIiIiIuXl5eWFx48fY9++fQAg0WchImns2xMR5a2QkBAEBwfj7NmzePHiBWbMmIHixYsXdLaISA6cWoSIiIiI8oyDgwNCQkJQsWJFuLm5/WfneSQiIiLlIEyFVKpUKbi6umL27Nlcu4PoF8FANhEREREREREREREpNc6RTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1AoXdAYoY+PGjUNsbGxBZ0NpXb58uaCzoPQqVKhQ0Fn4JcTHxxd0Fn4JbI/ko62tXdBZUHpv374t6Cz8Er59+1bQWVBKOjo6CAsLK+hsEGWbnZ0doqOjCzobSuvr168FnQWlp6enV9BZ+CV8//69oLPwS/Dw8CjoLPwSRowYUdBZUHoqKioFnYVfgpaWVkFnQSlpa2vj9OnTcu/PQLYSi42NRUxMTEFnQ2l9+vSpoLOg9EqUKFHQWfglMJAtH7ZHpChsv+UTFxdX0FkgIgWKjo5m+5eJL1++FHQWlF7hwrx9lwcfBMsnMTGxoLPwS4iKiiroLCg9BrLlk5ycXNBZ+C1wahEiIiIiIiIiIiIiUmoMZBMRERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZRERERERERERERKTUGMgmIiIiIiIiIiIiIqXGQDYRERERERERERERKTUGsomIiIiIiIiIiIhIqTGQTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1BjIJiIiIiIiIiIiIiKlxkA2ERERERERERERESk1BrKJiIiIiIiIiIiISKkxkE1ERERERERERERESo2BbCIiIiIiIiIiIiJSagxkExEREREREREREZFSYyCbiIiIiIiIiIiIiJQaA9lEREREREREREREpNQYyCYiIiIiIiIiIiIipcZANhEREREREREREREpNQayiYiIiIiIiIiIiEipMZBNREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZP/GKleujFmzZmH//v3w9vbGsmXLYG1tLff3LS0tMXz4cKxZswZeXl7w9vbG6tWr0aVLF6ipqcn8jpqaGrp164b169fj8OHD8PT0xOzZs1G9enVFHZbC1alTB/v370dkZCRevXqF06dPw9nZOVtpFClSBBMmTMDVq1fx9u1b3L9/H//88w9Kly6d4XfU1NQwfPhwnD9/Hi9evMCLFy9w8eJFLF68OJdHpHh//vkn1q5di+DgYISFhWHXrl1o3bq13N83MDDA0KFDsXLlSpw5cwa3b9/GyZMnM/2OqqoqnJ2dsWvXLgQEBCA0NBSHDx/G2LFjUapUqdweUp6oWbMmNm3ahKtXr+LmzZvw8vJC27Zt5f6+oaEhRo4cifXr1yMwMBCPHz/G+fPnM/3O48ePM/xbuHBhbg8pT9SuXRu7d+/Gw4cPER4eDh8fHzg5OWUrjSJFimDcuHG4ePEiIiIicOPGDSxevDjDc05LSwszZsxASEgIIiMjcffuXfz777+oWrWqIg5J4WrUqIENGzYgLCwMV69exb59+9CmTRu5v1+xYkWx/fb398f9+/dx5syZLL+noqKCDh06YNeuXQgLC8O1a9dw4sQJzJs3D8WLF8/NIeUJCwsLeHl5ITIyEq9fv8aZM2fg4uKSrTSE9vvatWt49+4dHjx4gBUrVsjVfvv7++Ply5d4+fIlQkJCsGTJktweUp6wtLTEsWPH8PHjR3z58gXBwcHo1KlTttIoUqQIpk6divv37yM2NhbPnz/HunXroKenJ7VvsWLFMHbsWOzcuRN37txBYmIikpKSYGRkpKhDIvpPqlOnDvbt24fw8HA8f/4cp06dQvv27bOVRpEiRfD333/j0qVLePXqFe7evYtly5bJbPNWr16NqKioTP/Gjx+vqMNTmPxu88zNzTFr1iwEBwfj9evXiIuLw6NHj7Bq1SqUL19eUYelcObm5tixYwfu3buHx48f49ixY2jXrp3c3zcyMsK4ceOwdetWXLlyBa9evUJoaGiW32vatCkOHDiAhw8f4sGDB/Dy8kKTJk1ycyh5RrhPjIiIwMuXL+Hn55ej+0Q3NzdcuXIFb968wb1797B8+fIs+xnDhg3DuXPn8Pz5czx//lxp7xMBIDw8HIsXL8bgwYPRv39/zJw5E2FhYdlO58uXL9i1axf+/vtv9OvXD0OHDsWsWbPk6sNu3boVvXr1Qq9evRAdHZ2Do8h7lpaWOH78OKKiovD161cEBwejc+fO2UpDaJsePHiAuLg4vHjxAuvXr8+0P7Zr1y7cvXsXP378QHJyslL3x9hnlU+dOnWwd+9ePH36FJGRkTh58mSu+gMvX77EnTt3MuwPAEDRokUxdOhQnDt3Dk+ePMHTp0/h7++PsWPHQlNTUxGHJTeV6Ojo1Hz9xQLg4OCA4OBgpW3QMjJo0CDExMTk6Lu1atXCnDlz8OPHDwQEBOD79++wsrKCvr4+Nm3ahEOHDmX6fTU1NRw5cgSJiYm4desWIiIiUKRIEdStWxcVKlTAw4cPMWnSJCQkJEh8x8PDA9WrV0d4eDhu3rwJDQ0NWFlZQV1dHfPmzZOrcyOvixcv5joNa2trHDx4EPHx8fD29kZsbCycnJxgaGiIqVOnYvXq1VmmoaKiAi8vL9jZ2eHSpUsIDg6GqakpHB0dERkZCTs7O0RFRUl8R0tLCwcPHoSlpSVCQ0Nx+fJlAGkdQisrK5iamub62IC0YFZu1atXDxs2bEBCQgJOnjyJuLg42NnZoUKFCli8eDF27NiRZRrt27fH3LlzkZSUhGfPnsHExARv377NNDC3bNkytGzZEpGRkQgODkZiYiJq1aoFCwsLvH//Hl26dJEq15yKj4/PdRoNGjTAli1bkJiYiOPHjyMuLg6tW7eGgYEBPDw8sGXLlizT6NChAxYuXIikpCQ8ffoUZmZmePPmDZo3b57hdx4/foyXL1/C29tb6jN5g5fyyml7lJ6VlRX27t2LhIQEHD58GHFxcXBwcEDFihUxc+ZMrF+/Pss0VFRUsHv3brRo0QJXrlxBSEgIKlWqBHt7ezx//hwODg4SdUNHRwfHjx+HqakpLl++jKtXr6JMmTJwcHBAcnIyOnbsiOvXr+f62NL/Xm7Ur18fmzZtQmJiInx9fREXF4dWrVqhQoUKWLhwIbZt25ZlGs7OzvDw8EBSUhLCw8NhamqKt2/fws7OLsPvqKmpYcWKFWjevDkePHiAS5cuITExEeXKlUODBg3QqVMnvHv3LlfHJnj9+nWu07C2toa3t7fYfsfExMDJyQlGRkaYMmWK3O33gQMHMmy/bW1tpdoZbW3tTNtvExOTXB+bIC4uLtdpNGvWDL6+voiPj4enpydiY2Ph4uICY2NjuLm5Yfny5VmmoaKiguPHj6N169YIDQ3FhQsXYGZmBmdnZzx79gxWVlb4+PGjuL+RkRGePn0KAIiIiICmpiZKlSoFU1NTREZG5vqYSpUqhUePHuU6Hfp1/ap9e0tLS3z69ClH323SpAm8vLyQkJAg9lnbtWsHQ0NDTJs2DWvXrs0yDRUVFXh6esLW1haXL1/GxYsXYWJiAgcHB0RGRqJ169YSbZ69vT1q1KghM63hw4dDQ0MDLVu2xLVr13J0TD/78uVLrtMoiDYvODgYDRo0wKVLlxAWFobExETUr18f1tbW+PDhA5o1a4aHDx/m+tgAQF9fXyHpNG7cGLt370ZCQgKOHDmCuLg42Nvbo2LFipg9ezY2bNiQZRpdunTB8uXLkZSUhMePH6NKlSp4/fo1GjZsmOF3OnTogFWrVuHjx484evQoAMDJyQm6uroYMmQIfHx8FHJ83759y3UaTZo0yfQ+cc2aNVmmoaKigv3794v9DOGcE/oZLVu2lHmfeODAAVhaWiIsLAyXLl0C8P9+hpmZWa6PTbBq1apcp3Hv3j0sWrQIampqaNiwIYoVK4bLly/j48eP6NatG+zt7eVKJzIyEgsXLsS3b99gbm6O8uXLIyEhAa9fv4aqqirc3Nwy/O7t27exaNEiqKurIyEhAatWrYK2tnauj03w119/5TqNZs2a4cSJE2LbFBMTgw4dOsDY2Bh///233G2Tj48PWrdujZCQEFy4cAGVK1cW26bGjRtL9cfCw8MBSPbHTExMFNIf+zlvufU79ll/poh62aRJE+zfvx8JCQk4dOgQYmNj4ejoCENDQ0yfPl3u/sC+ffsy7A+0adNGom0qXLgwjh8/DktLS9y6dUuMxVlZWaFmzZq4f/8+WrVqhe/fv+fomHR1dcV7KnlkK5AdGRkJc3Nz2Nra4uDBgzL3CQwMRLt27dC3b1+5Klp+yG1n99WrV/jzzz/h4eGBoUOHAgBsbW2hqamJw4cPy/zO169fsWDBAhw9ehTv37+Hvr4+nJ2dMXHiRGhoaMj1uzkNZBcqVAgbN25E6dKlMW7cOLHxKl68OP755x/o6+tj4MCBeP/+fYZpqKqqolOnTvDx8UFsbKzE9ilTpqBhw4bYvHmzRD3o0KEDBgwYgMDAQCxcuBApKSkAgLJly2LVqlX48eMH+vfvn+PK/bPcBrJVVVVx+fJllC9fHi1btsTt27cBACVLlsTZs2dhaGgIS0tLvHjxItN0evToIY5aHzhwoLhdOAe2bt2KsWPHSnxn165dsLe3x6BBg3DgwAGpfCUnJ+fq2AS5DWSrqqri6NGj0NfXR48ePcROuIaGBvbs2YMKFSrA0dERb968yTQdAwMD6Orq4uHDh0hISMCVK1fw8ePHDAPZNWrUwN69e3Hr1i389ddfSEpKEj+bOHEievbsiTVr1sgV9JRHbgPZqqqqOHXqFMqWLYvOnTvj/v37ANLK6eDBgzAwMEDLli2zDN5VrFgRpUqVwv3795GQkIA7d+7gw4cPWQayw8LC0LNnz1wdgzxyG8hWVVVFUFAQypUrBwcHB9y9excAoKmpiRMnTqBixYqwsrLCy5cvM02na9eu+Oeff+Dt7Y1hw4aJ23v37o1FixZhx44dmDBhgrh9/vz56NevH9avX4+ZM2eK2+vWrYsjR47g6dOnaNasGVJTFfNcNzeBbFVVVfj4+KBs2bLo2rUrHjx4ACCtLu3fvx8VKlRA27Zts6xLBgYGKFWqFB48eICEhATcuHEDHz9+zDSQPWnSJPz1119YunQp/v33X4nPhM6posoot4FsVVVVXLlyBeXLl4ednZ1E+33u3DkYGhqibt26crXfa9euhZeXFwYMGCBu79evn9h+jxkzRuI7u3btgoODAwYOHJin7TeQ+0C2qqoq7t69CwMDA1hZWeHmzZsA0sopJCQExsbG+OOPP/D8+fNM0/nrr7+wefNm7N27F7169RK3Dxo0CGvXrsXGjRslzsUSJUqgUaNGuHr1Kj5//izedDGQrXzYt8/fvn1OA9mqqqoIDQ1F+fLl0bp1a9y5cwdA2vXzzJkzqFixIurXr5/l9bN79+5YtWoVDhw4gMGDB4vb+/Tpg6VLl2Lbtm1yjbA2NzfHuXPncPfuXdjY2GT7eDKS20B2QbV5w4cPx8mTJ8VgiMDNzQ0eHh7w9fXN9ptnGVFEIFtVVRUBAQEoV64cnJycJPpjPj4+MDAwgLW1NV69epVpOoaGhihdujTu3buH+Ph4PH36FB8+fMgwkK2lpYWLFy8iOTkZrVu3Fu8fypUrh1OnTgEAGjVqpJCHuLkNZKuqquLSpUvifaJwzpUsWRJnzpyBoaEh6tWrl2U/o3v37lizZg0OHDggdZ+4bNkybNu2Teo+cefOnbC3t8fgwYPzvJ+R20B2cnIyJkyYgM+fP2PGjBniKNZv375hxowZ+PjxY6ZvTAq+f/8Od3d3JCYmYtKkSTA0NJT6HVVVVZnf/fbtG9zd3WFqaoqYmBg8ePBA6QLZqqqquHfvHgwMDNC4cWOJtik0NBTGxsaoVq1alm1Tnz59xLYp/b3f4MGDxbZJuKYB0v0xX19ftG7dWikD2b9rn/Vnua2XqqqqCAkJQfny5dGmTRuJ/sDp06dRsWJFNGjQIMv+QLdu3bBq1SocPHhQqj+wZMkSbN++XaI/0L59e2zevBnHjx9Hnz59JNLasWMH7O3tMWzYMOzfvz9Hx5XdQDanFpFDYGAgAIgdtS9fvuDGjRsZdtyEEYZr165FlSpVMGzYMFSuXBmrVq2Ck5OTQkZ/ZkZ4gunv7y8GsYG0Rt7T0xNqamqwtbXNNI3k5GTxKdjP24XKWbNmTYnPhE7Lrl27xCA2ALx9+xZ+fn7Q1tZWqtfGbGxsYGJiggMHDohBECDtRmXZsmVQV1dHt27dskynd+/eAIDZs2dLbN+6dSuePXuGzp07o2jRouJ2S0tLODo6Yv/+/VKdEwAK7ZzkVv369WFoaAhfX1+JkSSxsbH4999/UaRIEbleYXn58iVu3bolMYI/MwYGBgCA0NBQiSA2AFy4cAFA7ke9KlKjRo1gZGSEY8eOiUFsIK2c1q9fjyJFiqBDhw5ZpvPixQvcuHFD7nL61TRp0gSVKlXCoUOHxJsmIC1AvnLlSqirq6NLly5ZptOjRw8AaQHq9Hbs2IGIiAh07NhR4pxr3bo1kpOTsWjRIon9r169Cj8/P1StWhWNGzfOzaEpTIMGDWBkZITjx4+LQWwgrS5t2LABRYoUkeuV1pcvX+LmzZty16UyZcqge/fuuHLlilQQG0gLYCsqiK0ITZs2hYmJCby8vKTa76VLl8rdfgs3JrNmzZLYvmXLlgzb73bt2sHT01Pp228AaNGiBczMzLB3717xhgD4f0BOXV1dvIZlRgjyT5kyRWL7xo0b8fTpU3Tv3l2inOLi4nDmzBl8/vxZQUdCpFi/Wt/e2toaJiYmOHjwoHjTCqRdP5cvXw51dXV07do1y3SEm/o5c+ZIbN+2bRuePXuGTp06SZzLGRECKLt27crOYeS5gmrz1qxZIxXEBoClS5fi27dvCg32K4KVlRUqVaqEw4cPS/XHVq1aBXV1dbmmO3j+/DmuXbsmd/13dHSEtrY2tmzZIjEI5s2bN9i6dStKlSqVrSn58lL6+8T051z6+0R5zjmhn5HRfeLP59yvdJ8IpI3Gfv/+vXgvJChevDicnJyQlJQktreZOXPmDKKiouDq6ioVxAaQYRAbSAv8JyYmKmTUdF5RVNvUv39/AMDkyZMltm/YsOGX74+xzyqf/OoP/Hw/LZzfZ8+elUrr9OnTAJDlAytFYiBbDkFBQShVqpQ4z3NwcDCSk5MznG96xYoVuH37NsaMGQNvb2/MnDkT3t7eGDNmDK5duybXUP/cqFWrFgDIfM1P2PZzEDo7hAvozxdSIbAo69VzYZu5uXmOf1fRhKD6uXPnpD4TTlArK6tM01BXV4elpSUePXok84n8+fPnoaGhgTp16ojbhIDm4cOHoauri549e2Ls2LHo0qWLUgVngbRpRQDZo9+Dg4MBpI1qVTThZqBhw4YoXLiwxGfCzUBO5l3LK/Xr1weQ1lb8TOi8CWWZF0qWLAlXV1cMGTIE3bp1Q5UqVfLst3JDCBb7+/tLfSbMBd6oUaNM01BXV4eFhYU4pcrPAgICUKJECYm2pkyZMvj06ZPM0TnCU/2szvX8ItSlzM45S0tLhf9u69atoaamhpMnT6J48eJwdHTEwIED0aFDB5QpU0bhv5dbQvstaw55of3O6sGpotrvcePGKWX7DaQF/IH/dzDT8/PzA4AsAyzq6uqoX78+Hjx4IHMUzJkzZ6ChoZEn9ZIor/xqffvM2jyhHytPn7Vu3boZXj/9/f2hoaGB2rVrZ5pO0aJF0bFjR8THx+d45FVeUbY2LzU1FT9+/JAalFHQhP5YQECA1GdCHy2r/lhOCGkKg1LSE/KS2bQk+UmR51xG/QzhnEvfzxDW+Thy5Ah0dXXRo0cPpb1PBCAO4JE1BZEQb0g/MCMjYWFhUFFRQb169fDmzRv4+fnh+PHjuHbtWqbnz7Vr1xAUFITevXtDS0srh0eR9zJrm4S3EeRpmxo0aPDb9seUrf1WVkK7k9n9dFYDtLLqDwQEBEj1B4TzWNaA2JYtWyIlJUVmLCSvFM56F8URnvIePXpUnHPZ0tISbm5uUhfLGzduYPfu3QgKCsKrV6+QmJgIExMTdO7cGSNGjJC52GBISAjmzJmDGzduQF1dHc2aNZN6wiBvPj98+CD+OzAwEDVr1sSzZ88AACdPnoS6ujq0tLQQHh6OwoULi08OU1NTsXPnTmhoaEjN4+Tm5oZ///0XO3bswLhx47KdL3kJi4rIenX78+fP+PbtW64WHmnZsiUA6UD5169fUaFCBejr60tdrIXX4CpUqJDj31U0YR5qWSMo3r9/j5iYmCznOq1UqRJUVVUlRr6nJ2w3NTVFSEgIAIgNgqmpKTZs2CBx0Y2JicGoUaOynMM8vwhP3mRdCKKiohAXF5cnCyE8fvwYO3fuRK9evXD48GEEBQUhMTER5ubmqF69OtasWSPzAURBMTY2BpA2r9bPPn78iNjYWHGfvPDHH39g7ty5EtsCAgIwYcKEHM/FmReE80loS9P78OEDYmNjUalSpUzTMDY2hqqqqsw00qdtYmIiPuz49OkTSpcujeLFi0sFs4W2W1Hz0ueWcD5lVJfy6pz7888/AaQ9FDlx4oRE8DoxMRHLli3D9u3bFf67OaXI9ltWGunTzqj93rhxo8z2W9Z89QVFmEfzyZMnUp+9e/cOMTExWc61aWpqClVVVZlppE/bzMwsXzuwpBzYt8+fvr3Qnsnqb75//x6xsbFZtnnC9TOjNk9I28TEJNM1bZycnMS1XpRtjnJla/M6duwILS0teHl5yZP9fCP0tXLTH1P07wrb8uJ3c0I4nzLrZ2TVd8zqPlFI28TERKqfYWJigg0bNqBkyZLi/jExMRg9erTS3CcCaW9eA2lTif5MW1sbRYsWzXJ9laSkJLx48QKamprw8/ODt7e3xFuAZcqUwZgxY6SmzIyJicGWLVtQt27dPHnwokiVK1cGkHaP+zOhbRL2yUhWbZOQduXKlX/J/piytd/KSpH9gaxiWOn7A35+fvDx8YGjoyPOnTsnMUe2sPDvrVu3cnxc2ZVvI7I/f/6MVq1aYdGiRdDW1kbfvn3h5OSEGzduoF27djh+/LjE/tu3b8fx48dRvXp19OnTB7169UJqaipmzZqFfv36SaUfEBAAJycnXL16FU5OTujTp484SXl2O1lHjx6FhYWF+BcZGQl/f3/x3zt27EBCQgLq168PCwsLODg4iN99+vQp3rx5gwYNGqBEiRIS6ZYoUQINGjQQVz3OK8LvZjS/2Ldv36TyJi9LS0u0bdtWXCk9vStXrgBImwusUKH/Vy19fX0x+J3T380LQsfg69evMj+PiYmR6DxklkZGc/oJaadPR3jlYtasWfD19YW5uTmMjIwwcOBApKSkYMOGDWJQqaAJcz5mNDdybGys3PNCZteiRYuwYMEClC9fHj169EDfvn1hYWGBkJAQhS5gqAjCKr2ZlVNereT777//onPnzqhXrx5q166Nzp07w9/fH02bNsXGjRslzsWCJpRBbs45edJIvx+QNnJGVVUVf//9t8S+derUEdumrH43vwj5/nlaJ0Fe1SVdXV0AwLBhw/DgwQM4Ojqibt26GDJkCD5//oxJkyZlOFKxICiy/c6qLqVPR1jtfPbs2fDx8YG5uTkMDQ0xYMAApWu/AYiB9syuUVmNYJInjfT70X8H+/b517dXZJuXUV9FVpsnizC9l7JNKwIoV5tnYGCAf/75R5wrWJlk1W+NiYnJk75GZvVY3vqXXwqqn5H+PtHHxwe1a9eGkZERBg0ahNTUVKxfv16p+hnC2lfFixeX+XmxYsWynK88NjYWKSkpiI2NxeHDh9G1a1esXr0aK1asgLOzMz58+IBly5YhMTFR4nvbt29HUlKS1Hy9yig/2iZlO4eyS5nab2VWUG0TkDZ/9ooVK1CjRg0MGTIEQ4YMQc2aNeHj4yNzhHheytGI7PDwcHh4eMj8LKPJ1ydMmID79+9j5cqVEnPbCAuZjRkzBnZ2duI8LOPGjcOSJUsk5kNKTU3FyJEjsWvXLoSGhoqvHqWkpGD06NFISkqCr6+v+EQuNTUVgwYNyvZTcGtra3Hkmb+/P7Zu3YolS5ZAT08P0dHRGD16NHr06IFWrVoBkGy40z9ZlcXExARnz57F06dPxXmAfxWVK1fGpEmT8O3bN8yfP1/qNZ/Dhw/DxsYGTZs2hYGBAW7duoUSJUrAysoK7969g4aGhsTc2f9VQmDx3r17EosxeHl5QVNTE8uWLcPgwYMxatSogspigVNRUcH06dPRtm1beHh44Ny5c4iPj0edOnUwadIk7N69G/369ZOY1++/auHChRL/vnHjBgYNGoSdO3eiQYMGsLOzE1/H+q9atGgRmjdvjmHDhqFu3bq4du0aypQpA0dHRzx69Ah//vmnUs3/XBCEdunTp08YPXq0ON9lQEAApk2bho0bN6Jv375yzXP4O8us/S5ZsiSWLVuGIUOGYOTIkQWVRaIcYd/+v9m3z65KlSqhcePGiIiIkDk9BKXR1dXFsWPHUKZMGfTp04eL05Lc0vcz0i9IJ9wnLl26FIMGDcLo0aMLKosKJ/TBU1JS0Lp1a9jb24ufdezYEW/evEFYWBguX74sTqsQGhqKsLAwDB48WKGLOhKRbMWKFcPGjRthYWGBQYMGidNBNW3aFPPmzYOtrS1at26d5QK4ipKjQPazZ8+kgieZiYqKgre3N2xsbKQmaNfT08PIkSMxceJE+Pv7o02bNgAg9eoIkBbcGjBgAHbt2gV/f3+xsxsSEoKIiAi0adNG4rUSFRUVTJs2Dd7e3tlaGMHQ0FB8nfDMmTMoXbo0+vfvDxUVFfj6+gJIW7hBmMs0vaye8mT19EMRhJHYGY1+Ll68eIaj/TJSuXJlzJ07FykpKZg6darMm5rv37/j77//Rvfu3dGoUSM4Ojriy5cv8PX1xaVLl7B48eJcr0auSLJGS6enqamZ5YifnPz/Fv775MmTUvufOHECy5Ytk5grrSAJ9SSjURkaGhp5UpddXFzQqVMneHh4SNysBgUFYdy4cTh48CBGjx6NQYMGKfy3c0LWKOD08qqcMpKamor9+/ejQYMGsLCwUJpAdlYjBeQ55+RJI/1+QNoiQm3atIGbmxtatGiBOnXq4PXr11i8eDFevHiBDRs24OPHj9k9nDwh5DujNx00NDTypB0VfjckJERq0aagoCAkJCQo1QggRbbfWdUlWe33iRMnpPb39fXFsmXLspxfNj8JdSWza1RWi9vIk0b6/ejXxL69cvftFdnmZdRXyeqNJyBtNHahQoWwZ8+erLJcIJShzdPV1YWfnx/+/PNPDB8+XCnLKqt+q6amZp606enr8c//H+Spf/lJnnMuqzLKTT8jo/vEpUuXKs19IpAW3AKQ4ajr79+/Z/lGtpAGAJnHVqdOHYSFheHZs2ewsrJCbGwstm/fjtq1a2e5HoqyyI+2SdnOoexShvb7V1BQ90BjxoxB27Zt0bNnT4n26fDhw0hISMDOnTsxbtw4jB07Vu5jyY0cBbJtbW1x8OBBmZ8FBgaiXbt2EtuuXbuG5ORkJCYmyhztIczB8vjxY7Gzm5iYiI0bN8Lb2xuPHz9GbGysxIg5YT4mAOJqnbImNTc0NESFChUyHE2SlaCgIDRu3BgqKioA0haDKV68uFJdQH4mzI1dvnx5qfmBdHR0ULx48WyNDBCC2IUKFcLUqVNlzu0kiIuLw6ZNm7Bp0yaJ7XZ2dgBkzwtVUNLPf5p+ZVwgbS4uTU1NmQtmphcREYHk5ORMR+mk/y0grQwsLCxkNqDCNnlWjM8PkZGRANLOo3v37kl8VqpUKZQoUQK3b99W+O8KnZLLly9Lffbo0SN8+fIF1apVU/jv5pQwn7GxsbHUKPHSpUtDQ0MjX+eMAiDOjZ3Rq34FQWjrK1WqJFUeenp60NDQwPXr1zNNIzIyEsnJyRnOoShs/3nOr7dv32L8+PFS+wvTjfzcBhQU4ZwzNjaWOudKly6NEiVK5EldEuqwrM5vamoq4uLi8mwaoZxI337fuHFD4rPstt8ZzXEpax5uedrv9DdkBS39XIA/l4e+vj40NTVltrPphYeHIzk5OcN5CTOb05B+HezbK3ffPv18lbL6rBoaGlm2ecL1M6M2L7N5N4G0kaJdu3ZFUlKSUgZngYJv84Qgdu3atTFixAip+yFlkX4+6p/78UJ/7Odrq6J+t3bt2qhUqZJUQCqz+bMLQvp1jvLqPlE4F9Ofc0+ePPll7hOB/8+N/fbtW6m+eXR0NOLj47Ocr7do0aLQ0dHB58+fZQa9hW3C1CJRUVGIjY3FjRs30KtXL5lpCm/GzZ07N0/Wlsmu9PNXZ9Q2Xbp0KdM0smqbMpuH+1dQ0O33r0KR/YGsYljp2yYhnidrbnHhjV1hgdf8kC+TpwoXqtDQUCxcuFDqTxhxmX5O5969e2Pq1Kn4+vUrXFxcMG7cOEycOBFDhgwBACQkJIj7CjfewpxSP0u/aFVWAgMD4eHhAQ8PD8yePRvPnj3D58+fxW3Hjx+HlpYWlixZAg8PD+zevVvi+zmZM1nRhA6JhYWF1GfCNnmDj0IQW1VVFdOmTcPDhw9zlKfmzZsDkL06dkEJDg4GALRo0ULqM2E1VmGfjMTHx+Pq1auoUqWKzJFGzZs3R2xsrERwTjjRq1atKrW/sC2nN2eKJsx7LutGUni16+rVqwr/XWHBJ1mrc6upqaF48eJS86QVJKHjIWtUgDCvcFYXXkUTRoXm5Zyd2SUsZNOsWTOpz4Q2QtgnI/Hx8bh+/ToqV64s8xXupk2bIi4uTq7AdKFChdC+fXv8+PFDai7XgiLUk8zOOeG8VCRhIQ9ZHT8dHR3o6uri1atXCv/dnBLaZqHepCe031kt4pKT9lt4lV7WgzRhm7K038D/8yvMBZ+eMIVCVtMDxMfH4/Lly6hWrZo4ojU9Ozs7xMbG5km9JOXFvv3/5UffPrM2T+jHytNnvXbtWobXz2bNmonBIVlatmyJcuXK4ezZs3jz5k02jyB/FGSblz6IPWrUKKxfvz6nh5HnhL5W06ZNpT4T+mhZ9cdyQuhr2NjYSH0m5CWzhUbzk6LOucz6GcI5J6uf8SvcJwL/7/sIDx/TE+IN8gw+ql69OgDI7GsK24TrgYaGBpo2bSrzTxiJ26hRIzRt2lRpBmFk1ja1bt1aYp+MxMfH49KlS79tf4x9VvkIiyxmdj8t7JORrPoDTZs2leoPCPGZUqVKSe0vnJvp+3F5LV8C2cLQ9BEjRiA6OjrDv0mTJgFIG+Vx8uRJ2NraIiwsDCtXrsS0adPg7u6Ojh07SqUvdBwzejX8/fv3cuc1KChI7IQvW7YMQFoHWNgWGRmJN2/eiP/+eUSCrCer6aV/uptXbty4gTdv3qBZs2YST1mKFy8OV1dX/PjxA2fPnhW36+jowMDAQGrkppmZmRjEnj59Oh48eJDlb8saiebs7Iw6deogODhYqZ4QBgQE4NmzZ+jUqZPE06OSJUti3LhxSEhIwL59+8Tt+vr6qFy5stSNijDn4vTp0yW29+3bF5UqVYKXl5fEq/pHjhzBx48f0blzZ/GiDaQ1Du7u7gDSXtFQBmFhYXjx4gXs7e0lOlQaGhoYMGAAEhMTcfToUXF76dKlUalSpVx3GoRGc+DAgWKjKRg2bBjU1NTyPTCcmZCQEDx//hzt2rXDH3/8IW7X0NDAkCFDkJiYKLHCuJ6eHkxMTHJdTlWqVEHhwtIv1tSpUwcDBw5EYmKizFcTC0pgYCAiIiLg4uIiMU2FpqYmRo0ahYSEBImpZMqUKQMzMzOpV1+FBaYmT54ssb13794wNjbGwYMHJc65woULS41eUVFRwYwZM1C5cmVs3rw5yxXV80toaCieP38OR0dHiY6/hoYGBg8ejMTERIn2QU9PTyHn3OXLl/HkyRM0atRIKoguvCKmTHXJ398fz549Q+fOnaXa7/Hjx8vdfm/btg0ApBbi6tevX47b7/TnekET5u3t1q0bzM3Nxe0lS5bEpEmTxFcBBWXLlkXVqlWlykkYVThv3jyJ7YMGDYKpqSn27NkjNSUN/d7Yt/+//OjbX7hwAc+ePUPHjh1Ro0YNcbumpibGjh2LhIQEeHp6ituFNu/n6+eOHTsAANOmTZPY3qdPH1SqVAkHDhzI8Fzu2bMnAEgF+pVJQbV5Ojo6OHXqFGrXro0xY8Zg7dq1eXF4ChMUFISIiAg4OztL9cdGjhyJhIQEHDhwQNxepkwZmJqa5noByGPHjuHLly/o168fypUrJ24vV64c+vbti6ioKJlTdxWE9PeJ6c85Rd8n/nzOHT169Je5TwSAP//8E2XKlEFISIj4ViGQNtXI0aNHUbhwYYmBPtHR0Xj9+rXUVCTCw4Fjx45JPACNjo7GqVOnoKKignr16gFIC6QNGDBA5p9Qr7p3744BAwbIDLoVBEW1Tf/++y8AYP78+RLbBw8e/Mv3x9hnlU9O+gOy7qez6g/8fD8tDNxzc3MT32gD0gaGTZw4EUDWg4gUKUdTi2SXhYUFVFRU5A48Ca8UtWrVSmJBGED202Hhf+DFixelFsh7/vx5tkaRubu7ixeJcePG4ciRI3jy5AlUVFRw6tQpuLq64sSJExLz9aVnamqKcuXKISwsDHFxcRKvx8TFxSEsLAxGRkZ5uhhMSkoKVqxYgTlz5mDRokUICAjA9+/fYWVlBX19fWzatEniBqBPnz5o2bIlli1bhjNnzgBIC5rMmzcPmpqauHLlCurUqSP1ymVsbCyOHDkisW3nzp24desWXr9+jdTUVNSqVQuVK1fGo0ePsGLFijw75pxITk7GqFGjcPDgQfj4+MDb2xuxsbFwcnKCoaGh1FzgM2bMQPfu3TFs2DCJm5w9e/bAxcUFnTt3hpGREYKDg2FiYoJ27dohIiICc+fOlfjdmJgYjB49Gtu3b8fp06dx9OhRREdHo2nTpqhevTpOnTqlNDcJycnJmDFjBjZs2IBt27bh5MmTiIuLg52dHSpUqIDFixeLU9kAaXMntW/fHlOnTpWoG9ra2uIUDkBaYFFHR0eibJYsWSLO57Rv3z44OTmhYcOGOHr0KIKDg5GQkIDatWujVq1aiIqKwpo1a/K+AOSUnJyMyZMnY8uWLdizZw+OHz+OuLg4tG7dGgYGBvDw8JBoh/7++2906NABEydOhLe3t7hdR0dHvOkH/l9O6ectXbBggTgSrn///mjWrBmuXLmCt2/f4sePH6hcuTKaNGmC1NRUzJo1S6lGbSQnJ2P8+PHYu3cvDh8+jMOHDyMuLg4ODg6oWLEiZs6cKbFAxJQpU+Dq6orRo0dLXJA9PT3h5OSEDh06wNDQECEhIahUqRLs7e0RGRkpNc+rnp4eAgICEBAQgOfPn0NNTQ3NmjVDlSpVcPr0aanOYEFKTk7GtGnTsGnTJuzcuRO+vr6Ii4tDq1atUKFCBSxcuFDinBs7dixcXFzg7u4ucWOjra2NCRMmiP8uXLgwtLW1JY510aJF4jmXkpKCKVOmYOvWrVi/fj3OnDmDt2/fwsLCAubm5rh7965SvSKdnJyMkSNHwtvbG76+vvD29kZMTAycnJxgZGSEKVOmSLXfPXr0wNChQ6Xa7w4dOmTYfs+ZM0fid2NiYjBq1Cjs2LEDZ86cwZEjRxAdHY1mzZopXfsNpJXT4MGD4evri/Pnz8PT0xOxsbFwcXGBsbEx3NzcJG48582bh7/++gv9+vUTO7hAWme3S5cu6NatGypVqoQLFy7A1NQULi4uCA8Pl7pBB9Lql3ADKfTTFi1aJK69sGXLlixHs5HyYt8+TX717ZOTkzFmzBh4eXnh+PHjYp+1Xbt2MDQ0xLRp0ySun9OmTUO3bt0wYsQI7N27V9y+d+9eODs7o1OnTjAyMsLFixdRqVIlODo6IiIiIsProZ6eHlq2bIl3794p1UPNnxVUm3fgwAHUqVMH9+/fh66ursw2ccWKFUozL2tycjLc3Nywe/duHDx4EEeOHEFcXBzs7e1RsWJFzJ49W+KNPnd3d3Tp0gVjx47F/v37xe06OjoSx6qmpgZdXV0sX75c3DZ79myx3/rlyxdMnToVq1atwsmTJ8XBME5OTtDR0cHQoUMlgpgFSZ77xPTn3PTp08X7xJ/POWHtH0NDQ/Gck+c+0c/PT7xPbNasGf744w/4+fkp1dQ+qqqq6N+/PxYtWoS5c+eiYcOGKFasGC5fvoyPHz+iW7du0NPTE/f39PREUFAQBg4cKDEyv0qVKmjbti1OnDiBKVOmoHbt2khOTsa1a9fw9etXdO7cWeLhx68mOTkZgwYNwokTJ+Dv7w9PT0/ExMSgQ4cOMDY2xt9//y3RNs2fP19sm4SHIUDagxGhbTI2NsaFCxdgZmYmtk0/ByWBtL6XMGJWuLYuXrxY7I9t3rxZKfpj7LPKJzk5WWyLjx07hkOHDiE2NhaOjo4wNDTE9OnTJdqmqVOniv2B9A/f9u3bB2dnZ3Ts2FGibcqoP7B8+XK0bdsWXbt2hbm5uTjLgI2NDapVq4anT5/m60PcfAlk6+vrw8XFBd7e3li5ciVGjhwpEcUH0l6Vrl69OooXLy6+ehMaGorBgweL+9y/f18cSZFeo0aNYGRkhFOnTiEkJERiZfM5c+ZkazGY9IKDg9GwYUMxrxcvXkSxYsVQt27dDL+joqKCXr16YdGiRVi8eDFmzpwpfiY0GOPGjctRfrLj1q1bcHNzQ8+ePWFjYwNVVVVERkZi69atcq0yXrx4cfGpjaWlJSwtLaX2effunVQg+/z586hVqxZq166N1NRUvHr1Cv/++y+OHj2KpKQkxRycAgUGBqJNmzZwd3eHi4sL1NTUcO/ePcyYMUPuUXWpqano3r07xo4dC1dXVwwbNgyfP3/Gzp07MXfuXERFRUl9x8fHBw4ODvj777/Rtm1bFCtWDE+fPsX06dOxZs0apKSkKPpQc+zy5cvo3bs3hg8fjtatW6Nw4cJ4/Pgxli9fjlOnTsmVRvHixdG+fftMt61du1YMqsXFxaFHjx7o168fmjdvjvbt20NVVRXv3r2Dp6cnNm3apDQjaAVhYWHo1q0bRo0aBQcHBxQuXBiPHj3C4sWLxYWkslK8eHF06NBBYluJEiUktq1cuVK8IThz5gw0NTXxxx9/wMrKCmpqavj48SN8fHywbdu2fJ+XWx7BwcFo37493Nzc0L59exQuXBgPHjzA3LlzpdqTjKSmpqJPnz4YOXIkOnXqhEGDBiE6Ohp79uzBggULpM65r1+/4tSpU6hXrx7s7OyQlJSEBw8eYNy4cdi7d6/EHK3K4NKlS+jZsydGjBiBtm3binVp6dKlco9UKl68OFxcXCS2lShRQmLb6tWrJRYDuXXrFlxdXTFixAg0bNgQJUqUwJs3b7BhwwZs3LgR379/V8jxKUpgYCBat26NyZMnS7TfM2fOlHhAlJnU1FR069YNY8eORdeuXSXa7zlz5mTYftvb28PNzQ329vYS7ffq1auVqv0G0kavN23aFDNmzECXLl2gpqaGO3fuwN3dXeINiMykpqbCxcUFEydORI8ePTB69Gh8+vQJW7ZswfTp02WOmhVuztJLP/I2ICBAaW4KKPvYt0+Tn337oKAgODg4YOLEiXBxcUHhwoVx//59zJo1S+4RmqmpqejZsydGjx6NLl26YMiQIfj8+TN27dqF+fPny2zzAKBr165QU1ODp6dnjss+vxREmyfMwfvHH3/IDJIAaQEoZQlkA2l138XFBePHj4eTkxPU1NRw//59zJ8/X+Jty8yUKFECXbp0yXTb0qVLJebD9vb2xqdPnzBy5Ei4uroiNTUVt27dwsqVK8XAiLIICgpC27Zt4e7ujg4dOqBw4cJiPyM794k9evTAmDFj4OrqiqFDh4r9jHnz5sk853x9feHo6ChxnxgeHo4ZM2Yo3X0ikDYtiLAIb1hYGJKTk2FgYABXV1dxAV95dO/eHQYGBjhz5gwCAwOhoqICIyMj9O3bV2Ys4lfj7+8PGxsbzJw5U2ybbt++DXd3d4kHRJlJTU2Fs7MzJk6ciJ49e2LMmDFi2zRt2jSZ/bGOHTv+Mv0x9lnlk74/4OzsLPYHZs+ena3+QK9evTB69Gh07twZQ4YMQXR0NHbv3i2zP/Dq1SvY2tpi9OjRsLW1xV9//YXU1FS8ePECq1atwj///JOv1ziV6Ohoue/gIyMjYW5uLteCMH379pV4Gvv582c4OTnh9u3bqF69OurXrw8tLS28evUK169fx9OnT/Hw4UPo6+sjOTkZrVq1wtWrV9GoUSPUq1cPL1++xIkTJ9CqVSscOXIE3bp1w7p168T0/f390blzZxQqVAguLi4oV64cLly4gHfv3kFLSwt3797NcvXO9D58+IDKlStj3rx5GD58OIC0+XqKFi2KY8eOZfpdYSTmnTt30KJFC5ibm+PmzZs4d+4cLCws4OPjI9diUIMGDRJXlSZpWc39Q5A5HxtJ+5VfL8pPbI/kI2tud5KUflQ5ZUxZRqYpm1KlSmVr0WrKGPv2+du3t7S0FBdjJmnKFOhVVvr6+gWdhV/Cz1NXkGyrVq0q6Cz8Ev7666+CzoLS+/lhNsmmra1d0FlQSrq6utmaOjZf5sgG0m7s/fz8MHv2bBQpUgReXl7YuHGjOBn7+vXrxeH8qqqq8PT0RM+ePREREYGNGzfiwYMHmDNnDmbNmiUz/WbNmuHIkSOoW7cujhw5gm3btqFixYo4ceJEjiqLECQVFtf69u0bbty4If47MyVKlICPjw+GDh2KR48eYfXq1Xj06BFGjBiBI0eOyNXRJSIiIiJSVuzbs29PRERElN+yNSKb8hdHZGeOI7KzxhHZ8uGIbPmwPZIPR2RnjSOy5cMR2bJxRDb9qjgiO3MckZ01jsiWD0dky4cjsuXDEdlZ44hs+XBEtmxKOyKbiIiIiIiIiIiIiCgnGMgmIiIiIiIiIiIiIqXGQDYRERERERERERERKTUGsomIiIiIiIiIiIhIqTGQTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1BjIJiIiIiIiIiIiIiKlxkA2ERERERERERERESk1BrKJiIiIiIiIiIiISKkxkE1ERERERERERERESo2BbCIiIiIiIiIiIiJSagxkExEREREREREREZFSYyCbiIiIiIiIiIiIiJQaA9lEREREREREREREpNQYyCYiIiIiIiIiIiIipcZANhEREREREREREREpNQayiYiIiIiIiIiIiEipMZBNREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZBMRERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZRERERERERERERKTUChd0BihjGhoaBZ0Fpaarq1vQWVB62traBZ2FX0J8fHxBZ+GXULgwLxny4HmXNZ5z8ilatGhBZ0Ep6ejoFHQWiHKE14fMsZ+RNbZ/8uH1Uz5FihQp6Cz8EkqVKlXQWVB6KioqBZ2FX4KWllZBZ0EpZbd/pBIdHZ2aN1khIiIiIiIiIiIiIso9Ti1CREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZBMRERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZRERERERERERERKTUGMgmIiIiIiIiIiIiIqXGQDYRERERERERERERKTUGsomIiIiIiIiIiIhIqTGQTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1BjIJiIiIiIiIiIiIiKlxkA2ERERERERERERESk1BrKJiIiIiIiIiIiISKkxkE1ERERERERERERESq1wQWeAMlavXj18+vSpoLOhtFRUVAo6C/SbMDIyKugs/BIiIiIKOgu/hIsXLxZ0FpRe48aNCzoL9AvT1dXF5cuXCzobRNnGvn3m1NXVCzoLSq9169YFnYVfgp+fX0Fn4ZcwcODAgs7CL2Hbtm0FnQWll5SUVNBZoF+Yjo4OAgMD5d6fgWwl9unTJ0RFRRV0NpQWA9mkKFpaWgWdhV8C2yP5pKamFnQWlB7rEhH9F7FvnzkGsrMWGxtb0Fn4JfA8k8/3798LOgu/BD6AzNqPHz8KOgu/BMawFINTixARERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZRERERERERERERKTUGMgmIiIiIiIiIiIiIqXGQDYRERERERERERERKTUGsomIiIiIiIiIiIhIqTGQTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1BjIJiIiIiIiIiIiIiKlxkA2ERERERERERERESk1BrKJiIiIiIiIiIiISKkxkE1ERERERERERERESo2BbCIiIiIiIiIiIiJSagxkExEREREREREREZFSYyCbiIiIiIiIiIiIiJQaA9lEREREREREREREpNQYyCYiIiIiIiIiIiIipcZANhEREREREREREREpNQayiYiIiIiIiIiIiEipMZBNREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZBMRERERERERERGRUmMgm4iIiIiIiIiIiIiUGgPZvzFLS0v4+Pjg8+fPiI2NRUhICDp37pytNIoUKYJp06bh0aNH+P79O169eoUNGzZAT08vw+90794dYWFhiI2NxadPn3Ds2DHUqVMnt4eTZywtLXH8+HF8+vQJMTExuHjxYo7L6eHDh/j27RtevnwpVzmFhoYiJiYGUVFROHr0qNKWE8tIPjVr1sS///6La9eu4datWzhw4ADs7e3l/r6hoSFGjRqFDRs2ICgoCE+ePIG/v3+28jBr1iw8efIET548QenSpbN5BPmDbVPWbt++jUGDBqFevXqoU6cOXF1dceLEiWyl8e7dO8ybNw8ODg6oU6cOrKys0L17dxw5cgTJyclZfn/mzJmoVq0aqlWrhg8fPuT0UPIU65J8WE5Evweey/KpW7cuDh8+jLdv3yIqKgoBAQHo2LFjttIoUqQIJk+ejDt37iA6Ohrh4eFYs2aNzHKqXLky1q5di9DQULx48QJfvnzBw4cP4e3tjebNmyvqsBTO1NQUkyZNwtatW7Fjxw7MnTsXjRo1kvv7tWvXxujRo7F8+XJs3boVO3fuxPLlyzFkyBCUK1dOan8tLS3069cP8+bNw8aNG7F7926sW7cO06ZNQ/369RV5aApTt25dHDp0CG/evMHHjx9zXJfc3d1x+/ZtfP78GeHh4Vi9enWGdWnNmjUICQnB8+fPER0djQcPHuDgwYNo1qyZgo5K8QwMDNCnTx/MmDEDs2fPxrBhw1CzZs0cp1esWDG4u7tjwYIF6Nu3r9TndnZ2WLBgQaZ/2f3/lB8sLCzg5eWFyMhIvH79GmfOnIGLi0u20ihSpAgmTJiAa9eu4d27d3jw4AFWrFiR6X2fmpoahg8fDn9/f7x8+RIvX75ESEgIlixZkttDUjjhnHv9+jU+fPgAf39/dOjQIVtpCOfcrVu38OnTJzx9+jTDc06WQ4cO4du3b/j06VNODiFfWFhYwNvbG69evcL79+9x/vz5HJXTpEmTcPPmTURFReHJkydYtWqV3OXk7e2NuLg4REVF5eQQckUlOjo6Nd9/NZ85ODggODgY0dHRBZ2VbDE1Nc1xpWjWrBlOnTqF+Ph47Nu3DzExMejYsSOMjY0xfvx4LFu2LMs0VFRU4OvrizZt2iAkJAQBAQGoXLkyXFxc8OzZMzRs2BAfP36U+M7kyZMxb948RERE4ODBg9DU1ETXrl1RpEgR2Nra4uLFizk6nozyl1vNmjXDyZMnER8fD09PT8TExKBDhw4wNjbG33//LXc5+fj4iOV04cIFmJmZieXUqFEjmeU0d+5cREREwNvbG5qamnB1dUWRIkVgZ2en0HLKrf9CGZmYmOQ6jYYNG2LLli1ITEzE8ePHERcXh9atW8PAwAAeHh7YvHlzlml06NABixYtQlJSEp4+fQozMzO8efNG7o6rlZUVtm/fjri4OJQoUULmOZobT58+zXUa/4W26cGDB7n6fmhoKAYOHIgiRYrA3t4eJUqUgJ+fH16/fo0JEyagX79+Wabx4sULdOnSBdHR0WjSpAmqVq2K2NhYnD17Fh8+fICLiws8PDwy/H5wcDD69++P4sWL49u3bwgMDJS7UyOPatWq5TqN/0JdUoTfsZxKlSqlkPaIfl3s2/8e5/LP1NXVc51G06ZNcezYMcTHx8PLywsxMTFwcXGBkZERJk6ciBUrVmSZhoqKCo4cOYJWrVohNDQUgYGBMDMzQ/v27REREQEbGxuJcnJxccHq1asRFhaG58+fIyYmBuXLl4ejoyO0tLQwY8YMLFy4MNfHBgBOTk4KSefPP//ElClTkJiYiIsXL+L79+9o0KABypQpgx07duD48eNZptG3b1/UrVsXT548wefPn5GUlAQDAwPUrl0bycnJ8PDwwN27d8X9TUxMMH36dDx+/Bjv379HbGwsSpYsibp160JbWxtnz57Fhg0bFHJ8x44dy3UaNjY2EnUpNjYWzs7OMDIywqRJk+SuS4cPH0arVq0QFhYm1iUnJydERESgadOmUnVp1apVmdalRYsW5frYBKNHj851GiYmJujXrx+SkpJw8+ZNJCYmokaNGtDR0YGPjw8CAwOznaarqyuqV68OdXV1PHz4EFu3bpX6zYzu3+rVqwctLS3s3r0bt2/fztEx/WzdunW5TsPa2hre3t6Ij4+Ht7c3YmJi4OTkBCMjI0yZMgWrV6/OMg0VFRUcOHAAdnZ2uHTpEoKDg2FqagpHR0dERkbC1tZW6hqjra2NgwcPwtLSEqGhobh8+TIAwMjICFZWVgq5DwaAHz9+5DoNGxsbHD16FPHx8Thw4ABiYmIkzrmVK1dmmYZwzrVs2VLmOdesWbNM74/79u2LlStXIjExESoqKtDV1c31cf2cv9yysbHBkSNHxHKKjY1F+/btYWRkBHd3d7nL6dChQ2I5BQUFwdTUVCyn5s2bZ1pOffr0kSinUqVK5eqYdHV1cefOHbn3z1YgOzIyEubm5rC1tcXBgwdl7hMYGIh27dqhb9++WL58udwZyUu57ey+evUKf/75Jzw8PDB06FAAgK2tLTQ1NXH48GGp/W/duoXDhw/jxo0b4tMNKysr+Pj4ZOt3c9rZVVVVxYMHD2BgYICGDRvi5s2bAICSJUvi0qVLMDY2RpUqVfD8+fNM0+nTpw+2bt2KPXv2oEePHuL2wYMHY/369diwYQOGDBkibjczM8O9e/cQHh6O+vXr4+vXrwAAc3NzhIaGIjw8HDVq1EBqqmKeneS2EVBVVcX9+/dhYGCARo0aSZRTWFgYjI2NUbVqVbnKacuWLdizZw969uwpbh88eDDWrVuHjRs3SpXT3bt3ER4ejgYNGkiUU0hICMLDw1GzZk2FlVNu/FfKKLcXcFVVVfj5+aFs2bLo1KkT7t+/DwDQ0NCAt7c3DAwMYGdnh9evX2eaTsWKFVGqVCncv38fCQkJuHv3Lj58+CBXIFtDQwMnTpzAjRs3oKuriwYNGihdIPu/0jblJpCdlJQEe3t7vH37Fp6envjjjz8AADExMejcuTNevXqFkydPokKFCpmmM2vWLOzduxeTJ09G7969xe1fv36Fs7MzXr9+jbNnz8pMJyYmBu3atYO5uTk+ffqEy5cvK10g+79Sl3Lrdy0nBrIVh3179u0B5WnzchvIVlVVxa1bt1ChQgXY2Njg1q1bANLKKSgoCEZGRqhZs2aW5dS7d29s3LgRnp6e+Ouvv8TtAwYMwOrVq/Hvv/9ixIgR4vYiRYogMTFRKp1y5cohNDQUOjo6qFixIr58+ZKr4wMUE8guVKgQ/vnnH+jq6mLKlCmIjIwEkDYK1sPDA3p6ehg9enSWfUg1NTWZwasaNWpg+vTpePLkCSZPnixuV1VVRUpKilR9KVq0KObPnw8DAwOMGzcOL1++zPUx5jaQraqqips3b6JChQpo2rSpRF0KDAyEkZERatWqlWVd6tWrl1iX+vTpI24fMGAAVq1ahX///RcjR44Ut2dWl0JCQqCjowNDQ0OF1CUg94HsQoUKYdy4cdDS0sLatWvx5s0bAGnn8ogRI6Cjo4MlS5Zkq62uUaMGevbsicOHD8PZ2VlmIDsjGhoacHd3R3x8PObPny/XG4jyyG0gW1VVFVeuXEH58uVhZ2cnBthLliyJc+fOwdDQEHXr1sWLFy8yTadHjx5Yu3YtvLy8MGDAAHF7v379xDcjxowZI/GdXbt2wcHBAQMHDsSBAwek8qWoMsptIFtVVRU3btxAhQoV0KxZM4lz7sKFC+I5l1UZ9erVCxs2bICnp6fEaP4BAwZg5cqV2Lx5s8Q5l56hoSEuXbqEzZs3o0OHDtDX11e6QLaqqiquX7+OChUqoHnz5hLlFBAQACMjI5ibm8tVTuvXr8f+/fslyql///5iOY0aNUrmdw0NDREWFoYtW7bAxcUF+vr6+R7I5tQichCeItrY2AAAvnz5ghs3boj//pmPjw+WLVuGoKAg6Ovr51s+BS1atICZmRn27NkjdnSBtADG/Pnzoa6uLtEpy8jAgQMBAO7u7hLbN2zYgKdPn6JHjx4oWrSouL1v375QU1PDvHnzxI4uANy8eRN79+5F9erV0aRJk9wensII5bR3716pcvLw8JC7nISLSPqOGvD/curevbvMcpo/f77SlxPLSD6NGjWCkZERjh49KgaxASA2Nhbr1q1DkSJF5HrV58WLF7hx4wYSEhKynYdp06ZBXV0dM2bMyPZ38wvbpqyFhobi+fPncHR0FIPYAKCpqYnBgwfjx48fMoMsPxM6Lz9fp0qWLAkLCwsAwOfPn2V+d968eYiPj8f06dNzeBR5j3VJPiwnItnYt/89z+XmzZvD1NQUnp6e4s09kFZOixYtgrq6usSAiowIN/XTpk2T2P7vv/8iPDwcXbt2lSgnWYFHAHjz5g1CQkJQpEgRGBoa5uSQ8kSNGjVQtmxZBAcHi0FsAPj+/TsOHToENTU1NG3aNMt0Mgpc3blzB7GxsShbtqzE9uTkZJkPPeLj43Hjxg0AkPpOQWnWrFmWdSn9w6CMCG/R/dynykldCg0NVbq6ZGpqitKlS+PmzZtiEBsAEhIScP78eRQuXBh169aVO70SJUrA2dkZ165dw8OHD7OdHwsLCzHQp6gArSI0bdoUJiYm8PLykhgl/vXrVyxduhTq6uro1q1blukI7fysWbMktm/ZsgXPnj1D586dJeqTpaUl2rVrB09PT6kgNgClKiPhnNu/f7/UObd48eJst98ZnXOurq4SZZTe+vXr8fbtW8yZMycXR5K3MiunJUuWyN02CQ/Wfi6nzZs3/xLlxEC2HIKCglCqVClUr14dQNor18nJybC2tpa5v7OzM/z9/fHq1SscOnQoP7MKAOLoTT8/P6nPTp06BQBZdk7U1dXRoEEDPHjwQOaT5tOnT0NDQwOWlpYK/d38JE9+M7qhEWRVTmfOnJEqJ6EMZP2usE1ZyollJJ8GDRoASGsrfibcLOfl3H8tWrRAx44dMXv2bKWey4ttU9YuXboEIG2amJ8JwQLhlcDMVK5cGQBw4cIFie1fv37FtWvXoKenBzMzM6nvnTt3DocPH8bUqVNz/WQ9L7EuyYflRCQb+/a/57ks9EnPnDkj9dnp06cBIMP/xwJ1dXXUr18fDx8+lFlOZ8+ehYaGhlzBOV1dXdSrVw9xcXF49uyZPIeQL/78808AkHgoIhACysK5kROVK1eGhoZGliMCBWpqaqhRowZSUlLk/k5eE+rS2bNnpT4T6pc8dalevXq/dV0S3mp99OiR1GfCtkqVKsmdnouLC1JSUnI8or5evXoA/t+fVhZCH/78+fNSnwl1LKuHgurq6rC0tMSjR49knifnz5+HhoaGxPoFwkCqw4cPQ1dXFz179sS4cePQpUsX6Ojo5Ph48kJm7bewTZ4yEs45WWV07tw5aGhoiIN60hs6dCisra0xZMgQxMfH5+QQ8oXQ7iiqbcpJOTVp0gRDhw4t0HIqnJ8/FhMTg1WrVuHo0aOIiIhAkSJFYGlpCTc3N6mFJW7cuIHdu3cjKCgIr169QmJiIkxMTNC5c2eMGDECampqUumHhIRgzpw5uHHjBtTV1dGsWbMcPSWIiYmRWNgqMDAQNWvWFC8aJ0+ehLq6OrS0tBAeHo7ChQtLPBlNP4quIAgBjMePH0t99u7dO8TExIj7ZMTU1BSqqqoy00ifduXKlcXgXeXKlRETE4N3795lur+yEII4ylhOsgJMBYFlJB9jY2MAQEREhNRnHz9+RGxsrLiPomlra2PevHnw8/OTaz7DgsS2KWvCyCgjIyOpz/T09FC8eHGJ0VMZ6d+/P86fPw8PDw8EBgZKzJFdtGhRLFu2TOop++fPnzF9+nTY2dnB0dFRMQeUR1iX5MNyorzGvn3+4LksH1NTUwDAkydPpD4Tyimr/qOJiQlUVVVlppE+bTMzMwQHB0t8ZmZmhq5du0JVVRXlypWDg4MDtLW1MXLkSMTGxubkkPKEMOo5/QhawZcvX/D9+3eZizVmpFatWqhatSoKFy6McuXKwcLCAl+/fsX27dtl7l+yZEm0bt0aKioq0NLSQp06dVC6dGl4eXnJrGsFQagneVmXhCmyTE1NZdYlV1dXpa9LwqAHWVMmxcbGIiEhQe7F52vXro0aNWpgx44d+P79e4ajQTNibGwMPT09REZG4v3799n6bl4T2iZZ06K9f/8eMTExWU51WalSJaiqqmY4tVr6+hQSEgIgrUyFbRs3boSWlpa4f0xMDEaNGgVvb+9sH09eyKyMsnvOZVRG6dvv9Os7mJqaYvbs2eKivcpMnuucsE9GsionYbuscpo1axbWrVtX4OWUb4Hsz58/w97eHvfv30fDhg3Rt29fxMTEwNfXF+3atcO2bdskbpy3b9+OkydPonHjxmjZsiW+f/+OoKAgzJo1C9euXcPOnTsl0g8ICECnTp1QqFAhuLi4oFy5cggICECbNm0kTlh5HD16FMOHD5fYFhkZKfVEQhhhWbFiRYUtJKAIwvFmNHfW169fsywTedJIv5/w3xldNGTtX9BYTlljGclHU1MTADLsWMbGxor7KNqsWbOgpqam1NNACFifshYTEwMAGdYXDQ0NcZ/MlC5dGvv27cOECRNw4cIF8c2AokWLwtXVVeYc1bNnz8aPHz+UenoaAeuSfFhOlJfYt88/PJflI88xlixZUq400k+lkp5wDZaVTuXKlTF16lSJfQcNGoS9e/dmnfl8VLx4cQDAt2/fZH7+/ft3cR95mJubo127duK/37x5g3/++SfDkcNaWlro3Lmz+O+kpCTs3LlTIQs0Korw/zejuhQTE5PrupTZOZRRXdq3b1/Wmc9HQrA5o5GZ8fHxcgWkNTU14eTkhBs3buDevXs5yoswGlueNxfzm1BXMmtXsqpP8qSRfj8A4vo2s2fPxv79+7Fw4UJ8/vwZrVq1wtKlS7FhwwY8fPhQYlHWgqKIc06eNNLvB6TNWb1p0ya8ffsWM2fOzG62850816jc1iVh+8/ltHHjRqUppxwFssPDw+Hh4SHzs4wWPJgwYQLu37+PlStXSiw89eHDBzRv3hxjxoyBnZ2d2NCNGzcOS5YsgaqqqrhvamoqRo4ciV27diE0NBQNGzYEAKSkpGD06NFISkqCr6+vOAIkNTUVgwYNgpeXV7aOz9raWnyC7O/vj61bt2LJkiXQ09NDdHQ0Ro8ejR49eqBVq1YAkK0LPRGRotjb28PBwQF///23Qhd1pF9fZGQkhg4diuLFi2P37t2oVq0aYmJicPToUaxYsQJBQUHYvXu3eI319fXFiRMnsHDhQoUu6khEvwb27dm3p9w7ceIEihYtCjU1NRgZGaFfv37YvHkzLC0tMX78+ILOXp7ZuXMndu7cCXV1dRgYGKBTp06YM2cO1q1bJzXSGEhbx6NLly5QUVFB6dKlYWVlha5du6JKlSpYvnw5UlJSCuAolMuJEydQrFgxqbpUr16937IudezYEcnJyTh69GiOvq+uro6aNWsiISFBYt7g/7pChdJmEr537564sDEAeHl5oWTJkli2bBmGDBmS4eKH/wVjx45F/fr10aZNG3z//r2gs6O0hHJq27atUpRTjgLZz549w8KFC+XePyoqCt7e3rCxsZHo6AJpT4lGjhyJiRMnwt/fH23atAGQNhLiZyoqKhgwYAB27doFf39/sbMbEhKCiIgItGnTRuI1RhUVFUybNg3e3t7Zmsje0NBQfJ3wzJkzKF26NPr37w8VFRX4+voCSJtoPy/nvM0N4QlURiMkSpYsmeEiX9lJI/1+wn9nZ/+CxnLKGstIPsLTXQ0NDZmfa2hoZPjEM6e0tLQwc+ZMcU7jXwHrU9aEkdgZjbqOjY2Va/Sbu7s7Xr9+jdOnT4uB6RIlSmDQoEGIiorC9u3bxVGT0dHRmD17Npo2bYr27dsr7mDyEOuSfFhOJC/27dm3/x3OZXmOMTo6Wq40MhrRJlynM+vX/fjxA0+ePMHkyZNRvHhxDB8+HKdOnZI513hBEEZiZ/TAplixYoiLi8t2ugkJCXj69CkWL16MBQsWYPDgwbh161aGfZrU1FR8+PABhw8fRnJyMnr16gVbW1txPvOClNUbB5qamrmuS/KcQ+nrUrFixTBs2DClqkvCSOyMRl0XLVo0y6CXhYUFqlWrhl27dmX4lkBWzM3NUaRIEVy+fDnDBTMLkqwRrunJU5/kSSP9fun/+8SJE1L7+/r6YtmyZeL0IwVNEeecPGmk38/MzAxTp07Fxo0bZa51pYzkuUblti79PGLbzMwMU6ZMUapyytFij7a2toiOjpb5J+uVoGvXriE5ORmJiYnw8PCQ+rty5QoAyXnfEhMTsXr1arRo0QIVK1aEjo4OtLW1xUVH3r59K+57584dAEDjxo2lftvQ0BAVKlTIyWECSFsMpnHjxlBRUQGQthhM8eLFJSbRVzaZzVmnr68PTU3NDOfHE4SHhyM5OTnDee9kzdX3+PFjaGpqylzNPbO5/QqKMK+QMpZTRnOp5TeWkXyEubFlzYNdunRpaGhoyJw/OzfKly8PXV1dtGjRAk+ePJH4ExafDA0NxZMnTwp8bk8B26asCXNjy5oH+8OHD/j27ZvM+bPTi42NxbVr12BiYiJzdLVQP4RXN9+8eYPo6GgEBASgWrVqEn/C65nW1taoVq0a7t+/n6vjUxTWJfmwnEhe7Nuzb/87nMvp5/X8mVBOWfUfnz17huTk5AznYs1s7mRZhMW3slocPT8J55qsebC1tLRQrFgxmfNnyyslJQV3795F0aJFs5yrVSCMohUWoixo6efS/Zmi6lJmcwLLIizupkx1SZgbW9YC4RoaGlBXV8/yrdHy5csDAHr27IkFCxaIfxMnTgQAVK1aFQsWLMCoUaMyTENZF3kUpJ+/+mdlypSBpqYmwsPDM00jIiICycnJGZ5TsuqT0D7LelgibCtWrJgcR5D3Miuj7J5zGZXRz+33H3/8gaJFi2LIkCH49u2bxJ+RkRGKFi0q/ltZptGS5zqXVZuSVTn9PA93tWrVxHKKi4uT+BPKSfh3fpVTvsyRLYwQCA0NzXRS8PRPfnv37o2TJ0/CzMwMLi4u0NPTQ+HChfHlyxesX78eCQkJ4r7Ck4KMFhIoU6ZMhq9F/iwwMFB8yvDjxw88e/YMBgYG4uuWx48fh5aWFpYsWQIgrTPdo0cPudLOLwEBAZg8eTJatWoFT09Pic9at24t7pOZ+Ph4XLp0CY0aNYKhoaFU+bVs2RKxsbHijYqQZuPGjdGqVSupeQ7l/d38FBAQAHd390zL6cKFC5mmkVU52dnZSZXThQsXMiwn4ZVWZSknlpF8Ll26JK7g6+PjI/GZsGqwojtW0dHR2L9/v8zPmjVrhjJlyuDIkSNISEjI8qlsfmHblLV69eph48aNCA4OhoODg8RnwrVJ6Kxn5MePHwCQ4f/3T58+AQCKFCkCIG3B0E6dOsncNyAgAB8+fICjoyOKFi0KbW1teQ8lT7EuyYflRHmFffv8xXNZPhcuXMCECRNgZ2cnNf1My5YtAUBcMyIj8fHxuHz5Mho2bCiznGxtbREbG4urV6/KlSchWJyUlCTvYeS5e/fuwcXFBebm5hILeQH/Xxwup/MUC3R0dABA7jcnsrt/XgsMDMSECRNga2srVZfs7OzEfTITHx+PK1euoEGDBgqtS0I/TxmEh4ejefPmqFKlitSUHlWqVAGADOdKFzx//lzmvNZFihSBubk5oqOj8fjx4wz7tfr6+qhYsSLevn2LFy9e5OxA8lhwcDDGjx+P5s2b4+DBgxKf2draAkCWI13j4+Nx9epV1K9fHxUrVpQ61ubNmyM2NhbXr18Xt124cCHDtXGEbfJeS/NaYGAg3NzcYGdnhwMHDkh8Jpxz8pSRcM7JKqMWLVqIA36AtIFD27Ztk5lWx44dUaxYMezatQsAJPooBSkoKAhubm6wtbXNsJyy0zbJU07Pnz9XunLK0Yjs7BKG8I8YMSLD0R7R0dGYNGkSgLRRHidPnoStrS3CwsKwcuVKTJs2De7u7ujYsaNU+sLQ94ye9mVn1dqgoCAsXLgQCxcuxLJlywCkVQRhW2RkJN68eSP+e8+ePdkqi/xw9uxZPH36FN27d4e5ubm4vWTJkpg8eTISEhKwY8cOcXvZsmVRtWpVqVcLNm7cCABScyYOHjwYpqam2L17t8TCDlu3bsWPHz8wZcoUibTMzc3RrVs33Lt3T2leRQD+X07dunWTKid3d3e5y2nTpk0AgPnz50tsF8ppz549Mstp8uTJSl9OLCP5XLx4Ec+fP4eTk5PE6GcNDQ0MHToUiYmJOHTokLhdT08PJiYmGU5FIo83b95g8uTJMv+EDqOHhwcmT56cq1E1isS2KWuNGjVCxYoVcfz4cYnRzzExMdiwYQPU1NQkpv94//49wsPDJV7b1dHRQaVKlfD69Wupm6+vX79iy5YtAP4/MrtcuXKYO3euzD/hLYOJEydi7ty5MkdvFQTWJfmwnCivsG+fv3guy+f8+fMIDw+Hq6sratWqJW4vWbIkJkyYgISEBOzevVvcXrZsWVSpUkWqnITr5Jw5cyS2DxgwACYmJti3b59EOWU0mt/Q0BATJkwAAJw6dSp3B6dAt2/fxtu3b2FlZSXxllexYsXg4uKCHz9+SAxU0dbWRvny5aVGbpqYmMhM39zcHPXr10dsbCwePXokbjcyMpKYI19QokQJdOvWDQDEwElBU1Rd2rx5M4C0xfbSy0ldcnNzAwClmVYESBsdGhUVBXNzc4k+orq6Opo3b46kpCSJ/6eamprQ09ODurq6uO3WrVs4ePCg1N/JkycBAO/evcPBgwfFEek/EwZ4pH8Ip2z8/f3x7NkzdO7cGTVr1hS3lyxZEuPHj0dCQoLEQp76+vqoXLmyVH0Sgok/L8zer18/VKpUCV5eXhL16ciRI/j48SM6d+6M6tWri9vV1NTg7u4OABL3qAVJOOe6dOkidc65ubllu/3O6Jzz9PQUy+jWrVsYNmyYzL9Pnz4hKSlJ/HdGC5rmt8zK6e+//0ZCQoJEPyajctq6dSsA6XLq37+/zHIaPny4zD+hnIR/51c55cuIbAsLC6ioqMi9gqwQhGnVqpXUxS4kJERq/xo1agBICyb9/MrJ8+fP8erVK7nz6u7uLp7U48aNw5EjR/DkyROoqKjg1KlTcHV1xYkTJyTm61M2ycnJGDBgAE6dOoULFy5g3759iImJQceOHWFsbIzx48dLvLbu4eGBPn36oE+fPuJCOEDa6vKurq7o3r07KlWqhICAAJiZmaFDhw4IDw+XWEkZSHt1ZebMmZg3bx5u3ryJgwcPQlNTE127dgUADBw4EKmpqflTCHJITk7GwIEDcfLkSQQEBMDT0xMxMTHo0KEDjI2N8ffff0uU0/z589GnTx/07dtXqpy6dOkiltOFCxdgamqaaTnNmjULc+fOxY0bN+Dt7Q1NTU24uroCAAYNGqQ05cQykk9ycjImT56MLVu2YO/evTh+/Dji4uLQunVrcdRX+nbo77//RseOHTFhwgR4e3uL23V0dMSbfgAoXLgwdHR0JOYtXbBgQZbzYCortk1ZK1y4MObMmYOBAweiZ8+esLe3R4kSJeDn54fXr19jwoQJMDAwEPdftmwZDh8+jPnz56NDhw7idnd3dwwbNgzTpk2Dr68v/vjjD3z9+hXnzp3Dp0+f0KpVK5mv7P8qWJfkw3KivMK+ff7iuSyf5ORkDB06FMeOHcOZM2fg5eWFmJgYuLi4wMjICBMnTpQopzlz5qBXr14YOHCgxIjznTt3olOnTnB1dYWRkRECAwNhamoKZ2dnPHv2DDNnzpT43YULF8LExASXL1/GixcvkJKSAhMTE7Ru3Rrq6upYtmyZzHpeUFJSUrBhwwZMmTIFs2bNwsWLF/H9+3c0aNAAZcqUwY4dO/Dhwwdx/+7du6NZs2ZYs2aNxAj8BQsW4Pnz54iMjMSnT5+grq4OQ0NDVK9eHUlJSVi3bp3E6DwHBwdYWFjg4cOH+PjxIxITE6GnpwcLCwsULVoUISEhMheHLAjp69Lp06fh5eWF2NhYODs7w8jICJMmTZIYyTp79myxLgkjEwFg165dYl0yNjYW61L79u3x7NkzzJo1S+J3FyxYAFNTU6m61KpVK6irq2P58uVKV5cOHjyIfv36YfDgwbh58yYSExNRo0YN6OjowMfHR+K+pU2bNqhbty68vLzkHomeGVVVVdSpU0cqYK5skpOTMXLkSHh7e8PX1xfe3t6IiYmBk5MTjIyMMGXKFIn6NGPGDPTo0QNDhw6VCEru2bMHHTp0QOfOnWFkZITg4GCYmJigXbt2iIiIkHr4FhMTg1GjRmHHjh04c+YMjhw5gujoaDRr1gzVq1fHqVOnJILDBSk5ORnDhg3D0aNH4efnhwMHDiAmJibDc27WrFno1asXBg0alKtz7leTnJyM4cOH48iRIzh16hQOHDiA2NhYtG/fHkZGRnB3d5cqp549e2Lw4MFS5dSxY0d06dIFRkZGCAoKgomJiVhOPwe4lU2+jMjW19eHi4uLOAJDVofnypUr4uT+wmIwP7+qeP/+fXEkRXqNGjWCkZERTp06JdGwp6amYs6cOTl+RSk4OBgNGzYU59C7ePEiihUrhrp16+Yovfzk7++PJk2aIDg4GK6urhg6dCjevXsHV1dXmWUoS2pqKtq3b48ZM2ZAT08PY8eOhZWVFTZv3oxGjRrJHCUzf/589OjRAx8+fMDQoUPRpUsXBAYGonHjxlKvrikDf39/WFtbIzg4GF26dMGQIUPw7t07dO3aNVvl5OzsjJkzZ0JPTw9jxoyBlZUVtmzZgsaNG2dYTj179sSHDx8wZMgQdO7cGYGBgbCyslK6cmIZySc0NBRdu3bF1atX4eDggO7du+Pjx48YNWqUOBojK8WLF0fHjh3FP1VVVZQoUUJiW0YL8/wq2DZlrWHDhti9ezcsLCxw4sQJ7Nu3D6VLl8ayZcvQr18/udKwsbHB3r170aZNGzx58gQ7duyAr68vKlSogKlTp2L58uV5fBR5j3VJPiwnygvs2+c/nsvyCQgIQIsWLRASEoJOnTph0KBBePfuHXr27IkVK1bIlUZqaio6deqEOXPmQE9PD6NGjUKjRo2wbds2NG3aVKqc1q1bh9DQUJibm6Nfv34YNmwYLCws4OvrC0dHR0yePDkvDjVX7t69i2nTpuHhw4fi9DFfvnzB8uXLcfz4cbnS2LNnDz5//ozq1avD3t4eLVq0gLa2Nk6fPg03NzepB10XLlzA9evXUaFCBTRt2hQODg6oXr067t27h+XLlytd3+TChQuwtbUV69LAgQPx/v179OrVK1t1qXPnzpgzZw5Kly6NkSNHomHDhti2bRuaNWsmVZfWr1+P0NBQ1KpVC3379sXQoUPF/mC7du2Usi6Fh4dj/fr1iIyMhLm5ORo0aICYmBjs2bMnyykOcqt69eooUaIE7t69m+OFIvNLYGAgWrdujbCwMLi4uKB///748OED+vbti9WrV8uVRmpqKrp164b58+ejdOnSGDZsGBo0aICdO3fCzs5OnLM8PR8fH9jb2+PixYuwt7dH//79AQDTp09Ht27dkJKSotDjzI0LFy7Azs4OoaGh6Nixo8Q5t3LlSrnSEM65uXPniudco0aNsH37djRv3jzLOdt/BT+X04ABA/D+/Xv07t07W+XUpUsXsZxGjBghllOLFi2UvpxUoqOj5X6MLjROtra2UnP7CAIDA9GuXTv07dtX4mL0+fNnODk54fbt26hevTrq168PLS0tvHr1CtevX8fTp0/x8OFD6OvrIzk5Ga1atcLVq1fRqFEj1KtXDy9fvsSJEyfQqlUrHDlyBN26dcO6devE9P39/dG5c2cUKlQILi4uKFeuHC5cuIB3795BS0sLd+/ezdY8sR8+fEDlypUxb948DB8+HEDa3GpFixaVuehNeo8ePRKPPT4+HocOHUKZMmXE+Y8ASOQ9I6ampjIbI0oj3IQQ5VZGr0aSJHkXo/mve/DgQUFnQenJmquPSF6lSpVie6Qg7Nuzb69M0k83QLI5OTkVdBZ+CVmd05Rm9OjRBZ2FX4I87ft/nTLN3a7MGMOSTVdXV1zoWx75MrUIkPbKvJ+fHzZt2gRvb294eXkhJSUFZcqUQY0aNeDm5iaudquqqgpPT0/MnDkTZ8+exfXr12FiYoI5c+bAzs4OR44ckUq/WbNmOHLkCObOnYsjR46gaNGiaNq0KbZt24YhQ4ZkO7/CCAMrKysAwLdv33Djxg2MHz8+y+++e/cOe/fuldj2/v17iW1sDImIiIjoV8W+Pfv2RERERPktWyOyKX9x1Ebm+DSLFIUjsuXDEZDy4YjsrHFENuUGR2TTr4p9+8xxRHbWOCJbPhyRLR+OyJYPH1RmjSOy5cMYlmzZHZGdL3NkExERERERERERERHlFAPZRERERERERERERKTUGMgmIiIiIiIiIiIiIqXGQDYRERERERERERERKTUGsomIiIiIiIiIiIhIqTGQTURERERERERERERKjYFsIiIiIiIiIiIiIlJqDGQTERERERERERERkVJjIJuIiIiIiIiIiIiIlBoD2URERERERERERESk1BjIJiIiIiIiIiIiIiKlxkA2ERERERERERERESk1BrKJiIiIiIiIiIiISKkxkE1ERERERERERERESo2BbCIiIiIiIiIiIiJSagxkExEREREREREREZFSYyCbiIiIiIiIiIiIiJQaA9lEREREREREREREpNQYyCYiIiIiIiIiIiIipcZANhEREREREREREREpNQayiYiIiIiIiIiIiEipMZBNREREREREREREREqNgWwiIiIiIiIiIiIiUmoMZBMRERERERERERGRUitc0BmgjOnq6hZ0FpSaiopKQWeBfhPa2toFnYVfQqlS/2vnjlEYCKEoipJKBBvJ/vfoAoRkB5mEIfwnnFNP8QoRucU8qyccwd10zVniDu8jTuXsftZaq54Qb4xRPeEI3hnf6b1XTziCu/va3rt6Agebc/70/WOt9frTFgAAAAAAuM2vRQAAAAAAiCZkAwAAAAAQTcgGAAAAACCakA0AAAAAQDQhGwAAAACAaEI2AAAAAADRhGwAAAAAAKIJ2QAAAAAARBOyAQAAAACIJmQDAAAAABBNyAYAAAAAINobGRTb22Q3IbAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1600x200 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokens = [['[CLS]'] + glove_tokenizer.tokenize(sent) for sent in sentences]\n",
    "fig = plot_attention(tokens, alphas)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dNXPXiSlPIpa"
   },
   "source": [
    "# Contextual Word Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J5oqS0x7PIpa"
   },
   "source": [
    "## ELMo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "id": "5N8dJJJmPIpa"
   },
   "outputs": [],
   "source": [
    "watch1 = \"\"\"\n",
    "The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\n",
    "\"\"\"\n",
    "\n",
    "watch2 = \"\"\"\n",
    "Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\n",
    "\"\"\"\n",
    "\n",
    "sentences = [watch1, watch2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1M_qVWdNPIpa",
    "outputId": "e7af3e35-20f9-4b4a-af57-28bc426e3f9a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\""
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from flair.data import Sentence\n",
    "\n",
    "flair_sentences = [Sentence(s) for s in sentences]\n",
    "flair_sentences[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HsasoxvnPIpb",
    "outputId": "ad9b45ac-d715-4dcc-8b10-aa934c4607c8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Token[31]: \"watch\""
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flair_sentences[0].get_token(32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bdaPu5VNPIpb",
    "outputId": "4d4a6572-2ab9-4079-fe14-414dbdd91814"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Token[31]: \"watch\""
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flair_sentences[0].tokens[31]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "id": "g_iTjhAuPIpb"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, ELMO embeddings were returned by flair\n",
    "# using the allennlp library that has been archived.\n",
    "# from flair.embeddings import ELMoEmbeddings\n",
    "# elmo = ELMoEmbeddings()\n",
    "\n",
    "# As replacement, we'll be using flair's own embeddings instead\n",
    "from flair.embeddings import FlairEmbeddings\n",
    "flair_emb = FlairEmbeddings('news-forward')\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wZOGM9xpPIpb",
    "outputId": "2128428c-9248-410e-d42f-4136d02e009c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Instead of elmo, embeds sentences using flair embeddings\n",
    "# elmo.embed(flair_sentences)\n",
    "flair_emb.embed(flair_sentences)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UdFVdgRZPIpc",
    "outputId": "d66dc5e5-11c4-4b5c-b0f0-fb557f4effc2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(Token[31]: \"watch\", Token[13]: \"watch\")"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_watch1 = flair_sentences[0].tokens[31]\n",
    "token_watch2 = flair_sentences[1].tokens[13]\n",
    "token_watch1, token_watch2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3XbrpwUfPIpc",
    "outputId": "4b410d99-62dd-455d-d43f-aae5c1bed9e6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([-0.0007, -0.0057,  0.0187,  ..., -0.0051, -0.0022,  0.0026],\n",
       "        device='cuda:0'),\n",
       " tensor([-0.0011, -0.0037,  0.1031,  ..., -0.0079, -0.0044,  0.0002],\n",
       "        device='cuda:0'))"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_watch1.embedding, token_watch2.embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ucsg8w21PIpd"
   },
   "source": [
    "### Where do ELMo Embeddings come from?\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/elmo_lstm.png?raw=1)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/elmo_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "id": "AehQ4275PIpd"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, this piece of code only makes sense for\n",
    "# ELMO embeddings. Flair embeddings are structured differently\n",
    "# and therefore this code is commented out.\n",
    "# token_watch1.embedding[0], token_watch1.embedding[512]\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "id": "VFAgysccPIpd"
   },
   "outputs": [],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Unfortunately, this piece of code only makes sense for\n",
    "# ELMO embeddings. Flair embeddings are structured differently\n",
    "# and therefore this code is commented out.\n",
    "# (token_watch1.embedding[:1024] == token_watch2.embedding[:1024]).all()\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3hxX2iMHPIpd",
    "outputId": "fd30d170-0925-4cf9-f300-6239598fe3ea",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.5003, device='cuda:0')"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity = nn.CosineSimilarity(dim=0, eps=1e-6)\n",
    "similarity(token_watch1.embedding, token_watch2.embedding)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "id": "jF_w6p-rPIpe"
   },
   "outputs": [],
   "source": [
    "def get_embeddings(embeddings, sentence):\n",
    "    sent = Sentence(sentence)\n",
    "    embeddings.embed(sent)\n",
    "    return torch.stack([token.embedding for token in sent.tokens]).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kEU5nynEPIpe",
    "outputId": "0accaad5-cbb9-4c33-dab9-36226e8ec797",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-2.0933e-03,  5.0415e-04,  4.6944e-02,  ..., -4.4014e-04,\n",
       "         -3.9301e-02,  1.0601e-02],\n",
       "        [ 3.2224e-04,  1.4939e-03,  2.5947e-02,  ..., -9.2415e-04,\n",
       "         -1.4211e-02,  3.0942e-03],\n",
       "        [ 1.7407e-03, -1.8324e-04,  5.3375e-02,  ...,  4.5165e-04,\n",
       "          5.4484e-02,  9.7125e-03],\n",
       "        ...,\n",
       "        [ 2.2250e-03,  2.2138e-04,  4.4385e-02,  ..., -3.9766e-03,\n",
       "         -7.9449e-04,  2.2444e-02],\n",
       "        [-2.2455e-03,  9.7197e-04,  3.4624e-02,  ..., -7.1559e-05,\n",
       "          1.2647e-02,  1.3210e-02],\n",
       "        [ 1.0997e-03,  8.4726e-05,  1.5534e-01,  ..., -1.5737e-04,\n",
       "          9.5801e-04,  5.3785e-03]], device='cuda:0')"
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# UPDATED\n",
    "###########################################################\n",
    "# Instead of elmo, embeds sentences using flair embeddings\n",
    "# get_embeddings(elmo, watch1)\n",
    "get_embeddings(flair_emb, watch1)\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fX_xJposPIpe"
   },
   "source": [
    "## GloVe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "p7Y54X9_PIpe",
    "outputId": "45f4525f-a095-408a-98b4-95d29c3b12a8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-08-27 14:19:26,019 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim.vectors.npy not found in cache, downloading to /tmp/tmpk1h0brjh\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 153M/153M [00:09<00:00, 16.4MB/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-08-27 14:19:36,402 copying /tmp/tmpk1h0brjh to cache at /root/.flair/embeddings/glove.gensim.vectors.npy\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-08-27 14:19:36,724 removing temp file /tmp/tmpk1h0brjh\n",
      "2023-08-27 14:19:37,425 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim not found in cache, downloading to /tmp/tmpev4zf1hw\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 20.5M/20.5M [00:02<00:00, 9.83MB/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2023-08-27 14:19:40,243 copying /tmp/tmpev4zf1hw to cache at /root/.flair/embeddings/glove.gensim\n",
      "2023-08-27 14:19:40,267 removing temp file /tmp/tmpev4zf1hw\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "from flair.embeddings import WordEmbeddings\n",
    "glove_embedding = WordEmbeddings('glove')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mibBx6mvPIpe",
    "outputId": "c8faa86a-17ac-44c2-fb49-ccc037df0b45"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_flair_sentences = [Sentence(s) for s in sentences]\n",
    "glove_embedding.embed(new_flair_sentences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WzaqGrPBPIpf",
    "outputId": "9366b528-e815-444b-c1c7-212b10b30e43"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True, device='cuda:0')"
      ]
     },
     "execution_count": 146,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.all(new_flair_sentences[0].tokens[31].embedding == new_flair_sentences[1].tokens[13].embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mIH-ZF35PIpf"
   },
   "source": [
    "## BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 177,
     "referenced_widgets": [
      "afb9bfef678240eea5b8890258302834",
      "29a95b9254be492ba40595dbd4d56c2b",
      "7bb936c783ab402b870cf609dd9d3129",
      "589cc9d8dd0b40d5abb92bb83398f8c6",
      "6b5f3c6e7aa941799ea143fbaf6391ed",
      "099d406c76b44870be13ee5b3134a7de",
      "326894b89a9e471d8b3d13eb3c44ebdc",
      "5716cea9e8d84a289fe0760bcbc76d6a",
      "314323441385434a98cecb6fd4aaf2e3",
      "4801fffa2c324ebbac52ed45d6cc5be8",
      "97c86ac930a247b49bf111db20690f9f",
      "8d289b405a544e84a1694b0079a7ce4a",
      "eaa192d1608f4322854a8bf88242cfa7",
      "e2925c6848c0450b8150dbbc9197f7b4",
      "96787f87f23b4ceaa33955126aa9c252",
      "23cf666f7f614867a350e9ef44020040",
      "890bcfb8c23046808a22581da6107878",
      "110fcccc17cd464fb2b61838c02c45a0",
      "2b6f2219e25b4f70a21aab7dc67f210d",
      "0d47571efe944cb3bcd4062deef28a76",
      "8572ccf616654eca94d5699122aa51ce",
      "61f7bb49e16e486781de91a2975dc83e",
      "a1f68db900cd4f86a7add2b3866ce6e7",
      "4148fdd044b742829876144096e3d247",
      "717a6caa55834da9bdeb736a8bb9192f",
      "610cb20066c34d948550f9560e498430",
      "3721e8f28e1b4547aaedcd760bdfd69a",
      "b6f29e196ff944b8964a7ccf2a29aae5",
      "cbdcba4e4c4c4ec69224ebde008c16a8",
      "241a564f58564f3690a3955def8e997f",
      "11bea8fc0ead466bbb93aec568aaf1ae",
      "e34df6b7762b42169b6a38cb1db49c79",
      "cdad8f4cd3dd4e15be22b26eeb81813d",
      "089fea1d6a6744ae80454aa101f5898e",
      "5ddaead5423c4750aa5ceabb2f35e860",
      "cb0258347d984bdbac59ec33811f51b5",
      "efd84a7954324854a1d0fc0a533de65e",
      "1cd3ac215fd04246af43ad48f3a02633",
      "73f44d8a81bf480d831cc2630a054185",
      "b778db445dad4a328e2906d989745d53",
      "131e7124d505464f9e62e06ee7344fd9",
      "4d2c3a69a29243088ff2279457f4de62",
      "8736176520594608bfe10039ab53d571",
      "f898adb2fc6046329e9abae53870ce29",
      "b7f1693ebfac4297b33ff1e4f7091ff7",
      "4feb42875064419fb591079dbd104bfc",
      "980fde34ce5740a09517882962bca862",
      "849fd22e48d64438bb066a446c6bebee",
      "34d97e1f285448d8a67cdd3d3a52ab3c",
      "26e6170d5cd348dfa3a5de35f2e85ec3",
      "fa9f55afbab04780829b12fe4b116da3",
      "b3ad093b5a704c4dbe409b34bbbcb804",
      "632a507543cd43108c278887d0c0e0b3",
      "15d7e137a6ff45a89959fe0794b3894d",
      "7a9ecdfd864a4fcdb561b580069a7f2e"
     ]
    },
    "id": "wppmv3qLPIpf",
    "outputId": "02c69bbb-11a5-444f-c4ce-64582ed2e44a"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "afb9bfef678240eea5b8890258302834",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)okenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8d289b405a544e84a1694b0079a7ce4a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a1f68db900cd4f86a7add2b3866ce6e7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "089fea1d6a6744ae80454aa101f5898e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b7f1693ebfac4297b33ff1e4f7091ff7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading model.safetensors:   0%|          | 0.00/440M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from flair.embeddings import TransformerWordEmbeddings\n",
    "bert_flair = TransformerWordEmbeddings('bert-base-uncased', layers='-1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IXKlNCfdPIpf",
    "outputId": "d4d70e52-56db-4c5d-c077-b7d7b23b0002"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.6554, -0.3799, -0.2842,  ...,  0.0601,  0.8865,  0.4760],\n",
       "        [-0.1459, -0.0204, -0.0615,  ..., -0.0257,  0.5052,  0.3324],\n",
       "        [-0.0436, -0.0400, -0.0135,  ...,  0.3920,  0.5231,  0.9067],\n",
       "        ...,\n",
       "        [-0.2582,  0.6933,  0.2688,  ..., -0.0325,  0.0772,  0.2187],\n",
       "        [-0.1868,  0.6398, -0.8127,  ..., -0.5957,  0.2793,  0.1880],\n",
       "        [-0.1021,  0.5222, -0.7142,  ...,  0.3644,  0.0600, -0.1419]],\n",
       "       device='cuda:0')"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embed1 = get_embeddings(bert_flair, watch1)\n",
    "embed2 = get_embeddings(bert_flair, watch2)\n",
    "embed2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZNZPqZPUPIpg",
    "outputId": "39b74dcc-a3ad-428a-ff17-eff69b1732fe"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([ 8.5760e-01,  3.5888e-01, -3.7825e-01, -8.3564e-01,  1.3528e+00,\n",
       "          1.0204e-01, -8.2618e-01,  3.1570e-01,  3.1091e-01, -4.2653e-01,\n",
       "          1.6632e-01,  7.2439e-02, -1.0276e+00,  4.4680e-01, -1.4549e-01,\n",
       "          1.8315e-01,  3.7484e-01, -4.0245e-01, -1.4190e-01, -5.1596e-01,\n",
       "          1.1984e+00,  6.8202e-01,  1.1028e+00, -4.3955e-02,  1.6932e-02,\n",
       "          7.4420e-01, -1.0604e-01,  3.8719e-01,  7.3662e-02,  2.2424e-01,\n",
       "         -5.0281e-02,  1.9586e-01,  1.0773e+00,  1.0341e+00,  4.4217e-01,\n",
       "         -6.5567e-01, -5.6308e-01, -1.1827e-01,  1.2862e-01,  4.6581e-01,\n",
       "         -7.1305e-01, -1.5682e-01, -1.3552e-01,  4.5852e-02,  8.2849e-03,\n",
       "          5.2762e-01,  1.6906e+00,  5.5388e-01, -7.3777e-02,  5.1504e-01,\n",
       "         -2.0619e-01, -2.9505e-03,  2.5389e-01, -1.5380e-01,  8.4900e-01,\n",
       "          6.5440e-01, -3.2925e-01, -1.1198e+00,  2.4251e-01,  1.5586e-01,\n",
       "          5.7514e-01,  8.5833e-02,  3.3306e-01, -3.9344e-01, -4.1766e-01,\n",
       "          1.0790e+00,  1.1720e-02,  7.0214e-01, -6.0665e-01,  7.3702e-01,\n",
       "          6.8223e-02, -3.7110e-01, -7.7589e-01, -1.3057e-01,  1.7179e-01,\n",
       "          6.2051e-01, -6.4163e-01,  4.2881e-01,  5.7445e-01,  3.9801e-02,\n",
       "         -2.1279e-01, -5.1984e-01,  7.5919e-01,  3.0600e-01, -3.3987e-01,\n",
       "         -3.0427e-01, -5.7486e-01, -5.2273e-02, -6.8957e-01,  7.3935e-01,\n",
       "         -1.6546e-01, -8.2786e-01,  3.4169e-02,  7.1547e-01, -5.6963e-01,\n",
       "         -7.8686e-01,  3.6913e-01, -1.9961e-02, -5.7287e-01, -2.6407e-01,\n",
       "          8.5581e-01, -6.1419e-02,  3.8184e-01, -9.1567e-03, -1.0273e+00,\n",
       "         -5.3303e-01,  4.4014e-01,  1.7875e-01,  9.1851e-01,  5.9216e-01,\n",
       "         -4.8853e-01, -3.9106e-01, -8.8284e-02, -7.1935e-01, -3.8647e-01,\n",
       "          9.9550e-02,  2.2983e-01, -1.5325e+00,  6.5410e-02, -1.0896e+00,\n",
       "          1.2108e-01, -1.9921e-01, -2.5195e-01,  1.4421e-01,  2.7530e-02,\n",
       "         -3.0479e-01, -3.8533e-01, -1.1540e-01, -8.3979e-01, -7.7467e-01,\n",
       "          6.5181e-01,  3.1980e-01,  4.7616e-01, -8.2181e-01, -4.4323e-01,\n",
       "          1.2445e-01, -2.9082e-01, -4.2669e-01, -6.8031e-01,  1.1736e+00,\n",
       "          1.0199e+00,  7.3959e-01, -6.1510e-01,  1.0691e+00,  5.6806e-01,\n",
       "          9.4541e-01, -9.8110e-01, -7.1101e-01,  3.6151e-01,  2.0443e-01,\n",
       "         -5.0706e-02, -2.8712e-01,  1.2332e-01,  3.9614e-01, -4.4268e-01,\n",
       "         -5.5477e-01,  2.8325e-01,  5.5901e-01, -3.8944e-01, -4.2003e-01,\n",
       "         -1.2470e-01, -9.5830e-01, -3.4774e-02,  1.0381e+00, -2.2607e-02,\n",
       "          4.7267e-01,  8.1840e-01,  9.0778e-01, -1.9271e-02,  9.3009e-03,\n",
       "         -2.9097e-01,  5.1579e-01,  6.5791e-01,  3.1353e-01,  4.7983e-01,\n",
       "          6.2282e-01,  1.4813e-01, -3.8600e-01,  1.8680e-01, -7.3365e-01,\n",
       "         -5.2868e-01, -3.5756e-01,  1.4778e+00,  1.7065e+00,  5.2843e-01,\n",
       "          4.3788e-01, -6.0783e-01, -2.0891e-01, -2.6378e-01, -3.5563e-01,\n",
       "         -1.9136e-01,  4.4895e-01, -2.1979e-01, -2.2114e-01, -6.3080e-01,\n",
       "         -1.1537e+00,  3.8770e-01, -6.5095e-01, -1.4381e-01, -3.3387e-01,\n",
       "         -4.8165e-01, -2.2851e-01,  4.4550e-01, -1.4106e-01, -1.9820e-01,\n",
       "          1.2682e-01,  5.7340e-01,  5.2960e-01,  3.8535e-01, -1.8422e-01,\n",
       "         -2.7734e-01,  2.1105e-02, -5.1269e-01,  1.2723e-01, -5.2737e-01,\n",
       "          1.9794e-01,  6.6517e-01, -4.9908e-01, -5.2315e-01, -3.3081e-01,\n",
       "         -6.9502e-01, -2.1738e-01, -9.1546e-01,  8.4446e-01, -8.6271e-01,\n",
       "          9.8701e-01, -5.3786e-01,  2.2177e-01, -7.9477e-01,  1.6505e-01,\n",
       "         -3.9847e-01, -2.9883e-01,  2.0877e-01,  2.4684e-01,  1.0068e-01,\n",
       "          1.6014e-01, -1.1922e+00, -1.5679e-01, -2.0235e-03, -5.7468e-01,\n",
       "          4.6136e-03, -2.6884e-01,  4.2084e-01, -7.2789e-03,  3.4696e-01,\n",
       "          5.3609e-01,  2.1878e-01,  3.3520e-01, -2.8570e-01, -5.0958e-01,\n",
       "         -3.9454e-01, -8.9165e-01,  1.2181e+00, -6.8443e-01,  1.9280e-01,\n",
       "         -2.1449e-01,  4.6195e-02,  5.2594e-01, -4.3565e-02,  2.4555e-01,\n",
       "          1.9387e+00,  1.5278e-01,  9.9872e-01,  4.8577e-01, -4.1082e-01,\n",
       "         -8.3016e-01,  2.7045e-01,  4.8655e-01, -2.4722e-01, -1.8512e-01,\n",
       "          2.0524e-01, -6.6170e-02,  2.3728e-01,  1.2404e+00, -2.6077e-01,\n",
       "         -1.6077e-01,  1.3507e-01,  1.1847e-01, -2.9235e-01,  5.5437e-01,\n",
       "         -1.8560e-01,  1.3344e+00, -3.6667e-01, -3.7478e-01, -5.5665e-02,\n",
       "         -3.4930e-01,  8.5271e-01,  1.5741e+00,  6.0562e-01,  4.3780e-01,\n",
       "         -1.0298e+00,  4.0719e-01, -8.2617e-02,  2.7544e-02, -4.9199e-01,\n",
       "          7.3797e-01,  2.4101e-01,  9.6176e-01,  6.4367e-01,  1.1653e-01,\n",
       "          9.4047e-01, -2.6082e-01,  1.0976e+00, -1.1490e-01, -9.5253e-01,\n",
       "          5.8644e-02, -2.3060e-01, -6.7886e-02, -3.0703e+00,  2.4538e-02,\n",
       "          2.7582e-01, -4.3797e-01, -7.8420e-02,  4.2382e-01,  1.0763e+00,\n",
       "          1.8140e-01, -1.0572e+00, -3.9120e-01, -9.3492e-02, -3.7970e-01,\n",
       "          5.3947e-02,  1.1576e+00,  8.4130e-01,  1.8007e-01, -5.2631e-01,\n",
       "         -2.0291e-01, -1.9092e-02,  6.1720e-01,  1.1918e-01, -1.4457e+00,\n",
       "         -2.5592e-01, -1.2389e-01,  2.3928e-01,  1.1959e+00, -5.4284e-01,\n",
       "         -8.6952e-01, -4.8750e-01,  2.3500e-01,  1.3830e-02, -2.6659e-01,\n",
       "          9.7897e-01, -8.5489e-01,  3.1946e-01, -4.9998e-01, -3.2241e-01,\n",
       "         -1.3864e-01,  1.2900e-01, -7.5765e-01, -4.5315e-01,  2.9473e-01,\n",
       "          8.7803e-02, -1.3806e-01,  4.5430e-01,  8.0649e-02, -4.1994e-01,\n",
       "          5.8677e-03, -7.0909e-02,  3.8368e-01, -1.3059e-01,  8.7690e-01,\n",
       "         -6.1343e-01, -6.1015e-01, -7.0236e-01,  3.7512e-01,  3.3982e-01,\n",
       "          8.0857e-01, -1.6385e-01, -1.0569e-01,  6.1967e-01, -5.8364e-02,\n",
       "          2.3294e-01, -2.7713e-01,  5.2177e-01, -1.7158e-01, -1.0602e+00,\n",
       "         -4.3479e-01, -6.6953e-02,  3.6794e-01, -2.3135e-01, -2.0400e-01,\n",
       "         -5.3282e-01, -1.2286e+00, -2.4327e-01, -2.8658e-01,  1.3142e+00,\n",
       "          3.1902e-01, -9.1673e-02, -6.9225e-01, -3.0026e-01, -9.0491e-01,\n",
       "          8.4393e-02,  2.1298e-01,  8.2160e-01, -6.7558e-01, -8.9888e-01,\n",
       "          3.5113e-01, -5.2840e-01, -3.1060e-02, -2.8948e-01,  5.1492e-01,\n",
       "         -2.2329e-02,  1.6455e-01, -1.3406e-01,  7.9522e-01, -1.0633e-01,\n",
       "          4.0473e-01,  3.6492e-01, -6.1566e-01, -5.1250e-01, -5.4742e-01,\n",
       "          3.2135e-02, -4.5447e-02,  5.3155e-02,  7.0305e-01, -1.5488e+00,\n",
       "          6.2800e-01, -4.1181e-01,  3.1977e-01,  3.9906e-01, -1.7447e-01,\n",
       "          1.0357e+00, -5.4064e-01,  2.1086e-01, -5.4963e-01, -3.4769e-02,\n",
       "         -1.6814e-01,  4.8931e-01,  1.6012e-01, -6.6077e-01, -6.7637e-02,\n",
       "          9.2604e-02, -1.1114e+00,  4.6336e-01, -3.5745e-01, -3.2055e-02,\n",
       "          3.4265e-01,  3.5538e-01,  6.7498e-01, -2.0282e-01,  2.1987e-02,\n",
       "         -1.0548e+00,  9.8446e-01, -5.6733e-01,  6.9380e-02, -4.7736e-01,\n",
       "         -9.9546e-01, -3.5254e-02,  4.6173e-01,  7.1046e-01,  3.3707e-01,\n",
       "         -8.1136e-02, -4.8355e-01,  5.4515e-01,  3.7669e-02,  5.2528e-01,\n",
       "          4.1853e-02, -1.2564e+00, -3.2226e-01, -8.2649e-01, -7.5457e-02,\n",
       "         -7.5999e-01, -1.8107e-01,  1.1781e+00,  7.7896e-01,  6.6844e-01,\n",
       "         -7.6050e-01, -4.5029e-01, -5.4186e-01,  2.7285e-03,  2.3229e-01,\n",
       "         -6.9320e-01, -4.4790e-02, -7.4108e-02,  9.3096e-01, -4.1993e-01,\n",
       "         -7.3685e-01,  6.3003e-02,  4.0879e-01, -7.2932e-01, -1.6402e-01,\n",
       "         -2.7400e-01,  7.5049e-01, -6.5255e-01, -4.3226e-01, -6.2632e-02,\n",
       "         -8.2434e-01, -9.1996e-01,  5.1996e-01,  3.3342e-01,  3.0553e-03,\n",
       "         -3.7381e-01,  8.1910e-01, -2.5230e-01, -8.2635e-01,  3.8007e-01,\n",
       "         -1.0642e+00, -7.9937e-01,  1.0933e-01, -2.3652e-01,  4.5540e-01,\n",
       "         -2.9788e-01, -1.5564e+00, -2.3166e-01, -1.1862e+00, -2.9165e-01,\n",
       "         -2.1934e-01,  1.1169e-01,  1.3712e-01, -5.1983e-01, -2.5509e-01,\n",
       "         -4.4754e-01, -4.7948e-01,  4.6833e-01, -1.8625e-01,  6.9716e-01,\n",
       "         -1.1866e-01,  6.1084e-01,  6.3114e-01, -8.4579e-01, -4.6913e-01,\n",
       "          9.2668e-02,  2.9337e-02,  2.5508e-01,  4.5542e-01,  1.8251e-01,\n",
       "         -1.1855e-01, -1.2805e-01,  8.3519e-02,  1.7325e-01,  6.1112e-01,\n",
       "         -3.9921e-01,  1.6798e-01, -8.4275e-02,  3.9789e-01,  9.2255e-01,\n",
       "          7.7275e-02, -4.0163e-01,  3.1146e-01, -3.0563e-01, -5.8679e-01,\n",
       "          4.1130e-01, -4.3300e-01, -1.2496e+00, -8.5653e-01, -2.3101e-01,\n",
       "         -1.2980e+00, -2.7850e-01,  1.3634e-01,  3.4896e-01,  4.5002e-01,\n",
       "         -6.9828e-01,  6.2787e-01, -2.2598e-01, -1.7732e-01, -3.2117e-02,\n",
       "          4.8136e-01,  5.0901e-01,  5.0199e-01,  3.6993e-01, -4.5981e-01,\n",
       "         -6.7486e-02, -8.6722e-01, -5.7722e-01, -3.2765e-01,  7.8836e-01,\n",
       "          4.0974e-01, -1.8077e-01, -8.4644e-01, -2.5588e-01,  1.0655e+00,\n",
       "         -3.7329e-01, -4.9543e-01, -2.4806e-01, -3.7839e-02,  9.1717e-02,\n",
       "         -1.5725e-01,  2.4216e-01,  6.5864e-02,  1.3641e+00, -1.2707e+00,\n",
       "          7.8174e-02, -1.0401e+00, -3.9388e-01, -2.5627e-01, -4.9158e-01,\n",
       "         -4.0819e-01,  7.3981e-01, -4.4651e-01, -3.1208e-01,  6.2193e-01,\n",
       "          1.1230e+00, -4.4226e-02, -1.2044e+00,  2.2500e-01, -9.1115e-02,\n",
       "          3.5365e-01, -3.7870e-01, -1.0870e-01,  3.2082e-01,  1.2314e-01,\n",
       "         -2.1645e-02,  1.1413e+00,  1.3573e-02, -9.9737e-02,  9.3759e-01,\n",
       "          5.0417e-02, -4.9671e-01,  9.0127e-01,  1.4178e-01,  2.0027e-01,\n",
       "          6.3084e-01, -2.0751e-02,  3.0619e-01,  4.5247e-01, -7.6788e-01,\n",
       "         -7.2624e-01, -5.2419e-01,  6.0564e-01, -1.9887e-02, -2.9045e-01,\n",
       "          4.0384e-01,  9.0467e-01, -9.9711e-01,  1.0205e-02,  1.7958e-01,\n",
       "          3.5785e-02, -2.3696e-01,  6.4812e-01,  1.5352e-01,  3.1153e-02,\n",
       "          1.3805e-01, -1.4471e-01, -9.9713e-01,  1.7288e-01,  1.8074e-01,\n",
       "         -2.8294e-02,  7.8877e-01,  3.7700e-01, -7.0564e-02,  1.0712e+00,\n",
       "         -4.2392e-01,  4.1656e-01, -6.8028e-01,  1.6911e-01,  2.6617e-01,\n",
       "         -1.1312e+00,  1.6047e-01, -4.5070e-01,  4.9935e-01,  3.4413e-01,\n",
       "          6.4540e-01, -2.6727e-01,  1.5033e-01,  4.2149e-01,  3.4961e-01,\n",
       "         -1.5505e-01,  1.5471e+00,  7.7871e-01, -2.7912e-01, -4.5703e-01,\n",
       "         -7.9149e-01, -5.7845e-01, -2.4879e-01, -6.0028e-01,  6.9574e-02,\n",
       "         -1.9024e-02,  2.0336e-01,  1.3373e+00,  1.7257e-01, -3.3319e-01,\n",
       "          1.4024e-01, -2.2572e-02, -2.2282e-01,  3.4986e-02, -1.4985e-01,\n",
       "          5.2162e-02,  1.5562e-01, -2.3856e-01, -7.4213e-02, -2.1547e-01,\n",
       "          7.6899e-01,  1.1663e-01,  5.3913e-02, -2.6870e-01, -2.3951e-01,\n",
       "          8.8452e-01,  3.2750e-01, -7.8228e-02,  4.5901e-01, -4.3255e-01,\n",
       "         -4.4316e-01, -3.2505e-01, -1.4550e+00, -8.6595e-02, -7.3577e-02,\n",
       "         -2.0595e-01, -6.1714e-01, -7.1308e-01,  8.2567e-01,  1.6622e-01,\n",
       "         -8.5806e-01, -6.5291e-01, -1.5517e-02, -6.2837e-01,  5.6276e-01,\n",
       "          9.6670e-02,  4.9524e-01,  3.4259e-01, -6.7065e-01, -4.1591e-01,\n",
       "         -3.3399e-01, -7.4808e-01,  3.2285e-01, -5.1327e-02,  4.1144e-01,\n",
       "          2.6481e-01, -9.0313e-01, -4.4464e-01,  6.6254e-02,  4.1635e-01,\n",
       "         -7.5472e-01,  1.2165e+00, -4.2961e-01,  6.3139e-01, -2.1192e-02,\n",
       "         -4.6616e-01, -1.4575e-01, -7.4167e-01, -2.7592e-01, -4.6199e-02,\n",
       "          4.5393e-01,  4.7207e-01, -1.7622e-01, -6.7738e-01,  3.7787e-01,\n",
       "          5.3374e-01,  3.6832e-01,  1.3724e-01, -1.1172e+00, -2.5947e-01,\n",
       "          1.1888e+00, -1.0117e-01, -4.8335e-01,  5.8897e-02,  1.6610e-01,\n",
       "          6.6885e-01, -9.0577e-01, -2.3182e-01,  5.9258e-01,  1.1081e-01,\n",
       "          1.8942e-01,  6.5041e-01,  4.6973e-01, -3.1615e-01, -5.9470e-01,\n",
       "         -5.1061e-01, -9.7604e-01,  6.0229e-01, -6.3250e-01, -4.2595e-01,\n",
       "         -5.7413e-01, -7.1769e-01,  9.2507e-01, -6.2844e-01,  1.8694e-01,\n",
       "          2.0768e-01,  1.1880e-01,  4.1445e-01], device='cuda:0'),\n",
       " tensor([-9.8449e-02,  1.4698e+00,  2.8573e-01, -3.9569e-01,  9.6890e-02,\n",
       "          8.6436e-01, -3.7401e-01,  2.1752e-01, -2.5937e-01, -4.2487e-01,\n",
       "         -6.1410e-02,  3.3272e-01, -1.9102e-01, -2.9803e-02, -1.7782e-01,\n",
       "          7.1223e-01, -3.6092e-01,  4.7531e-01,  1.8901e-01, -1.9464e-01,\n",
       "          5.5041e-01, -2.0075e-01, -4.9742e-01, -1.3701e-02,  6.5578e-02,\n",
       "          2.2761e-01,  7.6017e-01, -7.9264e-02,  2.7941e-01, -5.4700e-02,\n",
       "         -2.6595e-01,  5.5095e-01, -2.1662e-01,  1.4398e-01, -3.8731e-01,\n",
       "         -1.9042e-01,  1.3155e-01,  2.2447e-01,  2.9488e-01,  2.7523e-01,\n",
       "         -2.7458e-01, -5.0479e-01, -3.9631e-01,  3.0837e-01, -6.3027e-01,\n",
       "         -3.5416e-01,  1.1856e+00,  8.9830e-02,  4.4822e-01,  4.0406e-01,\n",
       "         -3.4759e-01,  5.6113e-01,  1.0385e+00,  1.5012e-01,  3.7633e-01,\n",
       "         -3.9084e-01,  1.0627e-01, -5.0344e-02,  2.2779e-01, -5.2758e-01,\n",
       "         -1.9242e-01,  1.8374e-02,  5.2802e-01,  1.6686e-01, -7.3694e-01,\n",
       "          4.7303e-01, -4.7016e-01,  1.6289e-01,  9.7775e-02,  1.2824e-01,\n",
       "         -2.8870e-01, -3.9116e-01, -1.8324e-01,  5.2460e-01, -1.9830e-01,\n",
       "          5.4971e-01,  4.1160e-02, -3.3559e-01,  2.1169e-02, -8.5710e-02,\n",
       "          1.3109e-01, -3.6348e-01,  2.7382e-01,  1.1263e+00, -2.5038e-01,\n",
       "         -6.4161e-01, -3.2357e-01, -1.3938e-01, -1.0110e+00,  7.3331e-01,\n",
       "          1.0102e-01,  1.5387e-01, -2.9724e-01,  2.5424e-01,  3.2296e-01,\n",
       "         -3.5495e-01,  1.2272e-01,  9.6730e-02,  2.1486e-02,  9.1426e-02,\n",
       "         -1.4177e-01,  5.2628e-01,  2.1227e-01, -1.7098e-01, -3.8352e-01,\n",
       "          1.6601e-01,  4.4298e-01,  6.3219e-03, -5.9510e-01,  3.5965e-01,\n",
       "         -7.1792e-01, -3.6388e-01, -7.0435e-02,  1.8896e-01,  5.4332e-03,\n",
       "          1.1766e-02,  1.0752e-01, -5.5095e-01, -6.7836e-02,  3.5240e-01,\n",
       "         -2.6229e-01,  3.7230e-01, -8.1171e-02,  4.8515e-01,  1.0484e-02,\n",
       "         -2.9454e-01, -3.1111e-01, -2.6310e-01,  6.3381e-02, -1.0854e-01,\n",
       "          1.3688e+00,  2.9465e-01,  2.0623e-01,  3.8317e-02,  5.4437e-02,\n",
       "         -1.5726e+00,  3.7239e-01, -6.0964e-01, -8.8024e-01,  2.4682e-01,\n",
       "          6.7070e-01, -5.6562e-01,  2.1737e-01,  8.4505e-02, -2.6882e-01,\n",
       "          1.4112e-01, -8.0701e-01, -1.1845e-01, -3.2562e-01,  7.8287e-01,\n",
       "         -1.8123e-01, -7.1260e-01,  2.0073e-01, -2.2902e-01, -1.7190e-01,\n",
       "          8.2328e-02,  3.2728e-01,  3.3844e-01, -4.3359e-01, -2.0109e-01,\n",
       "          5.2177e-01, -2.1286e-01, -5.4271e-01,  4.2771e-01, -2.8440e-02,\n",
       "          6.8534e-01,  3.0166e-01,  3.3880e-01, -7.2538e-01,  4.7892e-02,\n",
       "         -5.7754e-01,  4.3418e-01,  6.2832e-01,  1.1230e-01,  1.6312e-02,\n",
       "         -3.1558e-01,  3.0021e-01, -3.4793e-01, -4.2287e-01,  8.9215e-02,\n",
       "         -1.7020e-01,  5.5911e-01,  4.3507e-01,  1.0352e+00, -6.0532e-02,\n",
       "          6.4033e-02,  6.3265e-01,  5.8860e-01, -6.2739e-01, -3.6926e-01,\n",
       "         -1.3001e-01,  3.0337e-01, -7.3679e-01, -1.0968e+00, -4.9620e-01,\n",
       "         -2.4885e-02,  6.0554e-02, -2.8982e-01, -5.5470e-01,  9.2082e-02,\n",
       "         -3.5928e-01, -6.8506e-01,  7.0259e-01,  7.4057e-01,  4.1953e-02,\n",
       "         -2.2261e-01,  3.1581e-02, -5.7034e-01, -1.5258e-01, -1.9746e-01,\n",
       "         -5.7486e-01, -6.6855e-01, -5.3475e-01,  2.1482e-01, -8.8653e-01,\n",
       "          9.2317e-02, -2.2077e-02,  9.8558e-01, -8.5093e-01, -1.6057e-02,\n",
       "         -1.1169e-01, -2.8778e-01, -4.2301e-01,  7.7859e-02,  6.9498e-02,\n",
       "          4.2390e-01, -7.0422e-02,  7.7604e-02, -4.0021e-01, -2.4131e-01,\n",
       "          1.3328e-01, -1.1587e-01,  9.0171e-01, -3.4312e-01,  6.0788e-01,\n",
       "          3.1653e-01, -2.4758e-01,  4.5196e-02,  4.9370e-01,  3.8995e-01,\n",
       "         -3.4692e-01,  3.3056e-02,  2.0106e-01,  4.0234e-01,  1.2993e-01,\n",
       "          2.9119e-01,  9.2159e-01, -4.1758e-01,  1.5004e-02,  5.2865e-02,\n",
       "          6.0922e-01,  7.6300e-02,  2.6940e-01, -6.3987e-01,  1.7187e-01,\n",
       "         -1.4997e-01, -2.2630e-01, -5.7578e-01, -5.4688e-02, -1.4096e-01,\n",
       "         -1.8318e-01,  1.6508e-01,  5.1326e-01,  4.4380e-01, -2.8080e-01,\n",
       "         -4.5417e-01, -3.0488e-01,  4.8724e-01, -1.7721e-01, -2.2289e-01,\n",
       "          6.0853e-01, -2.2074e-04,  5.6217e-01,  1.1224e+00, -1.4180e+00,\n",
       "         -3.6117e-01,  1.5190e-01, -1.7284e-01, -1.7632e-02, -2.1651e-01,\n",
       "         -8.6195e-01,  4.2031e-02, -4.6597e-01,  7.2990e-02, -6.2120e-02,\n",
       "         -3.9781e-01,  3.5995e-01, -3.4646e-01, -3.0336e-01, -1.5005e-02,\n",
       "          3.3628e-01,  5.7878e-02, -3.3315e-01,  1.1097e-01,  3.0777e-02,\n",
       "         -7.1115e-01,  3.6039e-01,  1.6007e-01,  3.4098e-01, -5.3309e-01,\n",
       "          5.6319e-01,  2.1157e-01,  2.6252e-01,  4.2321e-01, -5.2220e-01,\n",
       "          2.4814e-01, -3.6991e-01, -5.4816e-01, -5.3290e+00,  1.0130e-01,\n",
       "         -2.4280e-01,  2.6960e-01,  6.2188e-01,  3.4420e-01,  2.2724e-01,\n",
       "          1.2104e-01, -3.0778e-01, -1.3375e-01,  1.2070e-01,  6.9458e-01,\n",
       "         -1.7730e-01,  6.3572e-01,  3.9321e-01,  1.0834e-03, -4.1992e-01,\n",
       "         -3.1438e-01,  4.6218e-01,  3.0781e-01, -2.4236e-01, -6.3912e-01,\n",
       "         -7.2797e-03, -2.1637e-02,  2.9671e-01,  8.6041e-01, -1.6743e-01,\n",
       "         -5.2726e-01, -7.3631e-01, -4.9699e-01, -4.0285e-02, -5.5160e-02,\n",
       "         -2.8419e-01,  3.4066e-01,  1.9508e-01, -3.1847e-01,  1.1065e-01,\n",
       "         -1.3114e+00, -6.5144e-02, -5.2608e-01, -2.2926e-01, -4.9479e-01,\n",
       "         -3.6531e-01, -1.5899e-01,  9.3910e-02,  1.2114e-01, -1.5592e-01,\n",
       "          1.9144e-01, -7.1318e-01, -1.7203e-01, -6.8007e-02,  4.2680e-01,\n",
       "          2.7249e-01, -8.1867e-02, -7.7034e-01,  4.4867e-02,  6.1248e-01,\n",
       "          6.4211e-01, -4.2664e-01, -4.1450e-01, -4.6397e-02, -5.7337e-01,\n",
       "          4.2681e-01, -4.0819e-02,  2.8338e-02, -3.5921e-01, -2.4894e-01,\n",
       "         -4.9875e-02,  5.8249e-01, -1.4032e-02, -3.2829e-01,  1.0102e-01,\n",
       "         -4.5791e-01, -1.0023e+00, -2.6306e-01, -2.5233e-01,  6.0858e-01,\n",
       "          2.0588e-01, -2.5912e-01, -1.0158e-01, -3.4643e-01, -1.6082e-01,\n",
       "         -3.1902e-01,  5.3895e-01,  2.7880e-01, -6.8466e-01, -7.6330e-02,\n",
       "          9.4315e-02,  1.3531e-01, -8.8651e-01, -5.7966e-01, -3.3632e-01,\n",
       "          3.1987e-01, -1.9380e-01,  2.8006e-01,  5.9293e-01,  5.4404e-01,\n",
       "          3.7746e-01,  1.5074e-01,  1.3050e-01,  1.8917e-01, -8.2227e-03,\n",
       "         -3.1581e-01,  3.0006e-01,  2.2615e-01, -4.7198e-01,  2.9255e-01,\n",
       "          3.9098e-01,  7.1997e-01, -6.5656e-01, -5.7878e-02, -7.4665e-01,\n",
       "          4.5998e-01, -1.7215e-01,  3.4801e-01, -3.4560e-01,  3.4529e-01,\n",
       "          4.9162e-01,  3.4105e-01,  5.2297e-01, -5.5147e-01,  2.9069e-02,\n",
       "         -1.3466e+00,  1.6773e-01, -2.8781e-01, -6.5206e-01, -2.8571e-01,\n",
       "          4.4820e-01, -8.2600e-01,  2.7299e-01, -1.6052e-01, -4.9491e-01,\n",
       "         -4.4123e-01, -3.0832e-02,  3.5858e-01, -3.2000e-01, -3.6482e-01,\n",
       "         -4.6250e-01,  6.8682e-01,  6.2212e-01, -6.7616e-01,  4.4652e-01,\n",
       "          6.3600e-02,  5.6310e-01,  9.1847e-01,  4.4499e-01,  4.0972e-01,\n",
       "          6.3158e-01, -4.8508e-01, -4.0152e-01, -5.0366e-01, -4.5478e-02,\n",
       "         -4.5590e-01, -7.5448e-01,  6.8782e-01, -3.8171e-01,  2.5105e-01,\n",
       "          1.3606e-01,  1.9530e-01, -1.6731e-01,  1.8509e-01,  2.4151e-01,\n",
       "         -1.2497e-01,  7.3939e-02,  8.0915e-02,  4.0503e-01, -4.4470e-01,\n",
       "          1.4211e-01, -6.2765e-02, -1.0847e+00,  1.5061e-02,  3.1442e-01,\n",
       "          6.1791e-02, -2.7084e-01, -9.5111e-02,  8.0670e-01,  3.4657e-01,\n",
       "          1.7020e-01, -3.4658e-01,  6.2307e-01, -3.1217e-01,  6.7439e-02,\n",
       "         -4.2302e-01,  6.6040e-01,  2.8155e-01, -3.0293e-02,  1.3720e-01,\n",
       "         -4.0739e-01,  4.6180e-01,  7.8493e-01, -2.0656e-01, -6.3212e-02,\n",
       "         -8.2126e-01,  1.9584e-01, -8.8633e-02, -1.0430e+00,  2.6545e-01,\n",
       "          1.2379e+00,  6.7183e-01, -6.8933e-01,  2.3559e-01,  6.4715e-01,\n",
       "          1.2309e-01, -2.8892e-01,  2.7236e-01, -4.1748e-01,  9.3375e-01,\n",
       "          1.2763e-01, -4.8758e-01,  4.1386e-01, -5.7510e-01,  3.0247e-02,\n",
       "         -5.2484e-01, -3.5384e-01, -4.3309e-01, -5.0379e-01, -1.4411e-01,\n",
       "         -9.1641e-02,  1.3801e-01, -2.2797e-01,  4.0613e-01, -5.3039e-01,\n",
       "          2.8264e-01,  1.4536e-01, -3.8260e-01, -1.0636e-01, -1.9395e-01,\n",
       "          2.4888e-01,  3.9480e-01, -1.9706e-01, -9.8293e-02, -5.3219e-01,\n",
       "          3.9783e-01, -3.8129e-01,  1.8971e-01, -3.3756e-01, -6.2738e-01,\n",
       "         -7.0887e-01,  3.7720e-01,  2.1101e-01, -5.8501e-01, -4.8129e-01,\n",
       "          2.2947e-01, -1.3340e-01,  1.1180e+00,  6.6438e-02,  4.8484e-01,\n",
       "         -2.2995e-01, -3.2569e-01,  4.2933e-02,  3.4589e-01, -2.4830e-01,\n",
       "          1.0948e-01, -1.3069e-01, -5.7944e-01, -1.9196e-01, -7.8060e-02,\n",
       "          1.1934e-01, -2.3422e-01, -2.1563e-01, -3.3932e-01,  8.7396e-02,\n",
       "          2.8107e-01, -4.7970e-01,  2.1479e-01, -1.1446e-01, -5.0093e-01,\n",
       "         -1.2180e-01,  5.6778e-01, -1.7505e-01,  5.0499e-01, -6.7217e-01,\n",
       "          3.6737e-01, -1.8358e-01,  2.3660e-01,  2.9736e-02, -3.5624e-01,\n",
       "         -1.1451e-01,  6.5315e-01, -4.0913e-01, -5.6405e-01,  7.2596e-01,\n",
       "         -2.3398e-01,  1.7279e-01, -4.2138e-01,  1.0075e-01,  1.4012e-01,\n",
       "          1.6682e-01,  2.0844e-01,  7.5970e-02,  2.4352e-01,  1.3456e-01,\n",
       "          1.4141e-01,  1.7959e-01,  1.4143e+00,  1.2107e-01,  3.6410e-01,\n",
       "         -5.9567e-01,  4.5287e-02,  4.0946e-02, -6.3152e-03, -1.5476e-02,\n",
       "         -5.4967e-02,  4.0418e-02,  5.3073e-01,  3.6719e-01,  5.9931e-03,\n",
       "         -6.0457e-01, -4.0461e-02,  3.0526e-01,  2.2857e-01,  3.1699e-01,\n",
       "         -6.1115e-01, -3.5432e-01, -1.7680e-01,  3.0425e-01,  4.4631e-01,\n",
       "         -8.0750e-02,  3.3444e-02, -1.7067e-01,  2.9058e-01,  1.1439e-01,\n",
       "         -1.2962e-02,  2.6377e-01, -2.9394e-01, -2.3483e-01,  2.1922e-01,\n",
       "          7.9876e-01,  9.3168e-01, -1.6078e-01, -6.8044e-02, -2.8835e-01,\n",
       "         -3.7442e-02,  8.7288e-01, -3.0851e-02, -2.6066e-01,  8.7559e-01,\n",
       "          5.2805e-02,  4.2036e-01, -4.4657e-01, -6.0128e-01,  2.4042e-01,\n",
       "          1.0409e+00,  4.5662e-02, -5.9405e-01, -7.7239e-02,  4.4467e-01,\n",
       "          2.8607e-01,  3.1795e-01,  4.0720e-01, -4.4101e-01,  1.0156e-01,\n",
       "          8.4664e-02,  2.2385e-01,  3.2971e-02,  8.5590e-01, -8.3133e-02,\n",
       "          2.4421e-01, -2.5320e-01,  3.0961e-01,  3.2430e-01,  7.6288e-01,\n",
       "         -4.1389e-01, -7.4261e-01, -1.0888e-01,  3.0744e-02, -8.4961e-02,\n",
       "         -3.3006e-01, -9.4410e-02, -1.7684e-01,  9.1215e-02,  3.9247e-01,\n",
       "         -1.6165e-01, -2.5071e-01,  3.2119e-01, -5.8943e-01,  5.3371e-01,\n",
       "         -3.9494e-01,  4.5296e-01, -3.2192e-02,  2.9046e-02,  9.0094e-02,\n",
       "          8.0394e-02, -3.8424e-01, -2.4358e-01, -5.8893e-01, -1.6014e-01,\n",
       "         -5.0848e-01, -2.1314e-02, -5.0818e-01, -9.5969e-02,  4.0968e-01,\n",
       "          3.0001e-01, -5.3867e-01,  2.3648e-01, -5.9909e-01,  1.8138e-01,\n",
       "          5.6676e-01,  3.3343e-01,  3.6125e-02, -7.1760e-01, -2.8207e-01,\n",
       "         -1.7438e-01, -9.5262e-02,  2.1486e-01, -5.2020e-02, -1.4086e-04,\n",
       "         -1.5751e-01, -6.5503e-02,  3.1234e-01, -1.2235e-01, -1.2310e-01,\n",
       "         -2.6430e-01,  6.9013e-01, -8.8553e-01,  1.4780e-01, -4.3434e-01,\n",
       "          9.2149e-03, -5.1354e-01, -4.0582e-01,  4.0952e-02, -9.0218e-01,\n",
       "          1.7678e-01,  8.2160e-01,  9.0610e-02,  1.0791e-01, -5.0325e-01,\n",
       "          5.2987e-01, -4.0699e-01,  1.9170e-01, -2.1431e-01,  4.5255e-01,\n",
       "          5.6901e-01,  2.7348e-01, -4.4278e-01,  1.8978e-01,  1.3637e-03,\n",
       "          2.6983e-01,  6.8033e-01, -4.7854e-01,  8.3501e-01,  5.7284e-01,\n",
       "          5.5622e-01, -3.8540e-01, -1.7700e-02,  3.0426e-01,  2.9132e-01,\n",
       "         -2.5081e-01, -5.4555e-01,  4.0296e-02, -3.4137e-01,  4.2379e-01,\n",
       "         -5.2939e-01,  1.7819e-01, -1.2489e-02,  5.5839e-01,  1.0675e-01,\n",
       "          3.1746e-01, -2.8264e-01, -2.1325e-01], device='cuda:0'))"
      ]
     },
     "execution_count": 149,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_watch1 = embed1[31]\n",
    "bert_watch2 = embed2[13]\n",
    "bert_watch1, bert_watch2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qvTJj1vcPIpg",
    "outputId": "50579266-34fb-4276-bb68-be5a54878ee3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(0.3504, device='cuda:0')"
      ]
     },
     "execution_count": 150,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similarity = nn.CosineSimilarity(dim=0, eps=1e-6)\n",
    "similarity(bert_watch1, bert_watch2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qt9Wl8aIPIpg"
   },
   "source": [
    "## Document Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "id": "JgU-haTVPIpg"
   },
   "outputs": [],
   "source": [
    "documents = [Sentence(watch1), Sentence(watch2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eLbRmOR_PIpg",
    "outputId": "e0484999-c9f2-4bf1-f32a-77cd7ebab64c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence[58]: \" The Hatter was the first to break the silence. `What day of the month is it?' he said, turning to Alice:  he had taken his watch out of his pocket, and was looking at it uneasily, shaking it every now and then, and holding it to his ear.\",\n",
       " Sentence[48]: \" Alice thought this a very curious thing, and she went nearer to watch them, and just as she came up to them she heard one of them say, `Look out now, Five!  Don't go splashing paint over me like that!\"]"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from flair.embeddings import TransformerDocumentEmbeddings\n",
    "bert_doc = TransformerDocumentEmbeddings('bert-base-uncased')\n",
    "bert_doc.embed(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UB6GBlxYPIpg",
    "outputId": "43298176-16ce-4b58-a99f-8ac2f3d887ee"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-6.4245e-02,  3.5365e-01, -2.4962e-01, -5.3912e-01, -1.9917e-01,\n",
       "        -2.7712e-01,  1.6942e-01,  1.0867e-01, -4.0799e-01, -1.4945e-01,\n",
       "         4.4678e-02, -1.9687e-01, -3.0574e-01,  9.6215e-02,  2.8053e-01,\n",
       "         9.0462e-01, -3.0250e-01,  1.7854e-01, -1.6856e-01, -1.7227e-01,\n",
       "        -1.8270e-01,  4.5121e-01,  3.6129e-01,  1.2309e-01, -1.2047e-02,\n",
       "        -5.0255e-01,  4.2645e-01, -3.7184e-02, -1.7052e-01, -4.8920e-02,\n",
       "         1.0503e-01,  3.3456e-01, -3.8482e-02, -4.7287e-01, -7.4431e-02,\n",
       "         2.3990e-01, -8.3232e-02,  1.7974e-01,  4.8939e-01,  9.3024e-03,\n",
       "        -4.2722e-01,  2.6337e-01, -1.3217e-01, -1.3950e-01, -1.1859e-01,\n",
       "         4.5834e-02, -3.9031e+00, -1.0726e-01,  5.1104e-02, -9.9710e-02,\n",
       "         1.5913e-01, -9.5331e-02,  2.1513e-01,  4.7707e-01,  2.1226e-01,\n",
       "         4.3450e-01, -4.5154e-01,  1.8344e-01, -1.0171e-01, -3.6373e-01,\n",
       "         8.9560e-01,  3.5931e-01, -2.8519e-01,  2.9427e-02,  3.3968e-01,\n",
       "         4.1061e-01, -2.1051e-02,  2.7801e-02,  2.8888e-01,  4.5761e-01,\n",
       "        -3.8254e-01, -1.9211e-01, -2.6551e-01,  1.7064e-01, -2.0920e-01,\n",
       "        -3.3608e-01, -1.0044e-01, -2.9611e-01, -5.6573e-01,  4.3842e-01,\n",
       "        -8.9238e-02, -1.4645e-02,  2.2009e-01,  2.5755e-01, -2.6711e-01,\n",
       "        -1.4440e-01, -2.8944e-01,  2.2547e-01,  6.0303e-01,  1.0250e-01,\n",
       "        -2.0606e-01, -1.5392e-01, -3.2989e-03,  7.7604e-02,  1.8428e-01,\n",
       "         1.6114e-01,  2.5668e-01, -8.3616e-02,  2.7078e-01, -5.4946e-02,\n",
       "         4.9633e-01, -2.7734e-01,  4.1486e-01, -5.3346e-01, -7.8131e-01,\n",
       "         1.7047e-01,  2.8687e-01,  4.9673e-02,  4.6922e-01, -1.6500e+00,\n",
       "         4.6991e-01, -1.5328e-01,  2.7729e-01, -1.0157e+00,  1.5022e-01,\n",
       "        -1.7466e-01,  4.1232e-02, -4.4218e-01, -5.0299e-01,  1.1937e-01,\n",
       "         1.8505e-01,  1.7673e-01,  2.1935e-01, -1.3052e+00, -3.4139e-01,\n",
       "        -3.8845e-02, -2.4191e-01,  2.5906e-01,  1.2551e-01, -1.7339e-02,\n",
       "         4.4672e-01,  3.3219e-01, -2.1076e-01, -2.6042e-01,  2.1485e-01,\n",
       "         1.6804e-01, -1.6398e-01,  1.1147e-01, -1.4731e-01,  1.0370e-01,\n",
       "        -2.0025e-01, -1.7985e-01, -2.5822e+00, -9.3411e-02,  5.8352e-01,\n",
       "         4.2851e-01,  1.3719e-01, -4.4423e-01, -2.4912e-03,  6.3892e-01,\n",
       "         4.1029e-01, -3.2187e-01,  5.8879e-02, -2.1344e-01, -4.0472e-02,\n",
       "        -4.0414e-01,  3.8056e-01,  1.7888e-01, -4.1893e-01,  1.6880e-01,\n",
       "        -4.6562e-02, -3.3560e-01,  1.3599e-02,  2.4541e-01, -1.9909e-01,\n",
       "         5.3978e-01,  5.7400e-01,  5.7995e-01, -3.8400e-02, -4.4135e-02,\n",
       "         3.8312e-01, -1.6357e-02,  7.0812e-01, -1.6653e-01, -4.0108e-01,\n",
       "         8.2581e-02,  4.2481e-01, -1.2611e-01,  4.2772e-01, -7.9697e-02,\n",
       "        -7.4779e-01,  7.4372e-02,  2.0130e-01,  8.0436e-01,  8.2707e-01,\n",
       "         3.8594e-01, -2.6690e-02, -2.8838e-01,  2.1367e-01,  2.3883e-01,\n",
       "         1.4772e-01, -1.7737e-01, -1.6458e-01, -4.7632e-01, -9.9866e-02,\n",
       "        -7.7609e-02,  3.6381e-01, -5.1024e-01, -3.6723e-01, -2.4667e-01,\n",
       "        -2.0106e-01,  1.8970e-01,  1.2936e-01, -7.0922e-01,  1.8065e-01,\n",
       "         4.2441e+00,  3.2519e-01, -3.5622e-01,  7.9890e-02,  4.0457e-01,\n",
       "        -5.2027e-01, -4.0209e-01,  1.1543e-01,  5.4013e-01, -8.5186e-01,\n",
       "         5.7481e-01,  3.0116e-01,  3.0603e-01,  3.0002e-02, -3.7785e-01,\n",
       "        -2.3895e-01,  1.0855e-01, -7.3347e-02, -2.2568e-02, -1.8636e-01,\n",
       "         5.5137e-01, -2.5551e-01,  5.6108e-01,  8.9961e-02, -1.4174e+00,\n",
       "         1.9548e-01,  1.8128e-01,  5.7270e-01,  3.7286e-01,  2.1192e-02,\n",
       "         2.0576e-01, -2.7981e-01, -4.0403e-02, -5.4553e-02, -2.4785e-01,\n",
       "        -2.4466e-01, -4.0419e-03,  8.9292e-02,  5.5506e-01, -1.7438e-01,\n",
       "         4.4969e-01,  3.9400e-01, -5.3704e-01,  6.6374e-01,  3.8592e-01,\n",
       "         6.9357e-01,  6.0498e-01, -2.1726e-01, -1.0124e-01, -2.2627e-01,\n",
       "         5.7905e-01,  4.2297e-01, -1.2389e-01,  2.0031e-01,  4.5129e-01,\n",
       "         1.7262e-01,  1.0963e-02,  3.6436e-01,  5.7858e-01, -5.3210e-01,\n",
       "        -1.4753e-02,  6.1678e-01, -3.0254e-01, -3.7331e-01,  4.2933e-02,\n",
       "        -2.8411e-01,  2.5481e-01, -3.1135e-01, -2.4889e+00,  1.2208e-01,\n",
       "        -8.3915e-02, -1.0063e-01,  1.6028e-01,  9.8307e-02, -1.5816e-01,\n",
       "        -3.4995e-02, -1.4911e-01, -8.0463e-01,  5.4030e-01,  7.1562e-02,\n",
       "         1.7398e-01,  1.4491e-01, -7.0773e-01,  3.9191e-01,  1.0117e-01,\n",
       "        -2.8370e-01,  2.8936e-01,  1.5133e-01,  5.0763e-02, -2.4023e-01,\n",
       "         4.4110e-01, -3.9493e-02,  2.6435e-01,  3.1356e-01, -1.8297e-01,\n",
       "        -2.4273e-01, -1.7542e-01,  1.4501e-01,  3.0055e-01, -2.8845e-01,\n",
       "        -1.5901e-01,  1.7493e-01, -2.9559e-01, -3.4490e+00,  1.3099e-01,\n",
       "        -3.8233e-01, -1.0336e-01,  6.8291e-01,  6.9417e-01,  6.1574e-01,\n",
       "        -8.2423e-02, -1.2499e-01, -1.2243e-01,  2.8974e-01, -4.1564e-01,\n",
       "         6.8800e-02,  6.7222e-01,  5.1716e-01, -3.0253e-02,  3.6390e-01,\n",
       "        -5.9201e-01,  2.2436e-01, -2.7201e-01, -8.8382e-02, -5.5831e-01,\n",
       "         1.0460e-01, -3.3686e-01,  4.6233e-01,  6.3347e-01, -5.7720e-01,\n",
       "        -4.1045e-01, -6.3033e-02, -2.6181e-01,  2.5220e-01, -5.7010e-01,\n",
       "        -4.4242e-02, -2.5702e-03, -2.6010e-01, -2.0902e-01,  2.1671e-01,\n",
       "        -1.5338e-01,  2.3135e-02,  8.3758e-02,  2.4785e-01,  8.1829e-01,\n",
       "         4.6562e-01,  1.6161e-01,  1.5305e-01, -2.2000e-01, -2.5913e-01,\n",
       "        -3.0355e-01,  2.0841e-01, -8.6140e-02, -1.0182e-01,  7.8481e-01,\n",
       "         1.0315e+00, -5.8092e-01, -2.5592e-01,  1.4235e-01,  2.2627e-01,\n",
       "        -3.8135e-01, -1.7177e-01,  6.8986e-01,  9.5218e-01, -4.2518e-01,\n",
       "         2.9856e-01, -7.7871e-01,  4.6342e-01,  3.8082e-01,  3.3461e-01,\n",
       "        -3.6359e-01,  1.3449e-01,  6.0971e-02, -6.3682e-01,  6.5791e-01,\n",
       "        -3.4900e-01, -1.6919e+00, -1.1290e-01, -8.9322e-03,  7.0164e-01,\n",
       "        -2.4298e-01, -5.0133e-01, -9.4679e-02,  9.2998e-02, -6.9514e-01,\n",
       "         1.6732e-01,  1.0738e-01, -3.5135e-01, -4.1886e-01, -5.6835e-02,\n",
       "         1.8974e-01, -6.7744e-01, -5.0675e-02, -2.8237e-01, -9.4651e-02,\n",
       "         2.3333e-01,  2.1024e-01, -1.3943e-01,  4.7849e-01, -3.5598e-02,\n",
       "        -5.0289e-02,  2.2644e-01,  1.7870e-01, -8.6449e-02, -4.6139e-01,\n",
       "        -9.7895e-02,  9.5689e-03, -6.0332e-01, -1.0339e-01, -6.9934e-01,\n",
       "         8.6564e-01,  2.4269e-01,  1.2407e-01, -4.4824e-01,  6.2005e-02,\n",
       "         1.1447e-01, -4.3742e-02,  1.5308e+00, -4.7716e-01,  4.9675e-01,\n",
       "         8.2663e-01,  4.4054e-01, -8.6616e-02,  2.1151e-01, -1.6980e-02,\n",
       "         3.6449e-02,  5.2057e-02, -4.3879e-01, -2.0526e-01, -1.0826e-01,\n",
       "        -2.0654e-01,  5.4324e-02, -2.3006e-01, -2.3910e-01,  3.5886e-01,\n",
       "        -3.6865e-01, -4.1345e-01,  3.2852e-01, -2.1309e-01, -7.7967e-01,\n",
       "        -4.0635e-01,  4.1312e-01,  4.5644e-01,  5.6297e-01, -4.0113e-02,\n",
       "        -7.8324e-02,  3.5196e-01, -4.0508e-01,  4.9119e-01,  2.8871e-01,\n",
       "         1.2004e-02,  2.1122e-02,  5.8201e-01,  7.2554e-02, -1.2406e-01,\n",
       "        -2.2747e-01, -3.6875e-01,  1.2822e-01, -1.3219e-02, -2.5432e-01,\n",
       "        -5.4382e-01, -1.6104e-01, -4.9271e-01, -2.6982e-05, -1.6866e-01,\n",
       "        -2.0367e+00, -1.8150e-01,  7.2432e-01,  2.6644e-01,  2.3441e-01,\n",
       "         1.3906e-01, -7.0422e-03,  4.3320e-01, -3.9089e-01,  3.8913e-01,\n",
       "        -1.3505e-01,  1.8608e-01, -2.7714e-01,  7.7001e-02, -3.5178e-01,\n",
       "         3.1395e-01, -6.1126e-01, -1.9384e-01,  3.4077e-02,  7.6536e-02,\n",
       "        -6.3849e-01,  9.1558e-02,  6.1446e-01, -9.3747e-02,  2.7309e-02,\n",
       "        -2.9999e-01,  8.1415e-02,  2.3125e-01,  4.6289e-02,  1.2134e-01,\n",
       "         1.3592e-01, -5.6097e-01, -2.1794e-01, -6.8162e-01,  6.4019e-01,\n",
       "        -1.8568e-01, -1.6626e-01,  3.4131e-01,  1.8143e-01,  1.4511e-01,\n",
       "         4.1716e-02,  3.5290e-01,  2.6542e-01, -3.3044e-01,  6.5737e-01,\n",
       "        -4.8462e-01, -4.2369e-01,  5.4556e-01, -4.1632e-01, -7.4869e-02,\n",
       "        -2.7194e-01, -5.6729e-02, -1.9846e-01, -2.5328e-01, -8.4373e-01,\n",
       "        -2.5077e-01, -7.6245e-02,  1.3954e-01, -7.3810e-01, -7.1694e-02,\n",
       "         3.7180e-01, -6.6641e-01, -7.3948e-02,  1.0741e-02,  6.6874e-02,\n",
       "        -5.4790e-01, -2.3494e-01, -5.4339e-02, -2.7594e-01,  2.3641e-01,\n",
       "         9.0829e-02, -2.4923e-02, -6.5618e-01, -1.5407e-01,  6.5145e-03,\n",
       "        -1.8896e-01, -4.8536e-01,  4.1967e-01,  4.9990e-01, -2.3470e-01,\n",
       "         2.3999e-01,  2.2489e-02,  7.1898e-02,  3.7770e-01,  2.2666e-01,\n",
       "        -4.6463e-02,  7.9462e-02, -1.3600e-01,  2.6624e-01, -1.0557e-01,\n",
       "        -5.1542e-01, -2.1026e-01, -3.6529e-01,  3.7680e-01,  4.2558e-01,\n",
       "        -2.0673e-01,  1.7708e-01, -9.5000e-01,  3.5931e-01, -6.9485e-01,\n",
       "         2.6662e-01, -3.8393e-01, -1.4876e-01,  3.5606e-01, -1.9180e-01,\n",
       "         5.2490e-02,  6.9162e-01,  1.2743e-01, -2.9465e-01, -6.5748e-01,\n",
       "        -1.3319e-01, -1.6700e-01,  3.4043e-01,  1.0655e-01,  1.4711e-01,\n",
       "        -5.7975e-01, -1.9442e-01, -5.5820e-01,  1.5113e+00,  2.5854e-01,\n",
       "        -7.7182e-02,  1.9113e-01, -2.0155e-01,  5.2039e-02, -6.6810e-02,\n",
       "         7.7022e-01,  4.3337e-01,  2.1713e-01, -1.9559e-01,  1.1622e-01,\n",
       "         6.9701e-02,  1.9887e-01,  4.4560e-01, -2.0377e-01,  9.3891e-02,\n",
       "        -3.8858e-01, -1.0287e+00, -8.7257e-02,  3.0796e-01,  1.1579e-01,\n",
       "         2.2186e-01,  2.4396e-02, -2.2758e-01,  7.7906e-01,  8.9298e-02,\n",
       "        -5.7492e-01, -2.8990e-01,  5.5056e-01, -3.1180e-01,  5.1760e-01,\n",
       "        -2.1880e-01,  3.5772e-01, -9.8653e-01,  3.4399e-02, -1.5934e-01,\n",
       "        -2.6761e-01,  4.0133e-02,  7.4575e-02,  3.0935e-01,  2.3991e-02,\n",
       "        -2.4070e-01,  1.9046e-01, -3.6814e-01,  3.0126e-01,  1.0484e-01,\n",
       "         3.0939e-02,  3.2499e-01,  4.0934e-01, -1.7192e-01,  5.4189e-01,\n",
       "         3.5965e-01,  1.4977e-01, -2.0299e-01,  2.5857e-02, -2.3124e-01,\n",
       "        -5.2681e-01, -5.0653e-02, -4.0077e-01,  3.5184e-02,  6.1388e-01,\n",
       "         1.7713e-01, -5.8146e-01,  1.2276e-01,  2.9924e-02, -5.3960e-02,\n",
       "        -4.8978e-01,  8.5425e-02, -1.6308e-01,  2.2881e-02,  2.6261e-01,\n",
       "         2.0515e-01,  3.4557e-01,  2.7707e-01,  3.1900e-01,  6.0790e-01,\n",
       "        -5.5657e-01, -4.6023e-01, -1.5954e+00,  1.5275e-01,  3.5691e-01,\n",
       "         4.9637e-01, -1.7432e-01,  1.8391e-01,  5.2575e-01,  1.6172e-01,\n",
       "        -1.6158e-01,  1.2285e-01,  1.7329e-01,  1.0693e+00,  4.1782e-01,\n",
       "         2.6315e-01, -2.6412e-02, -3.4535e-01,  3.1312e-01,  3.1302e-03,\n",
       "        -1.3553e-01,  9.2502e-02,  1.8213e-01,  1.0432e-01, -5.5065e-02,\n",
       "        -5.2151e-01, -5.9766e-01, -3.0186e-01,  1.1254e-02,  6.2411e-01,\n",
       "        -2.4277e-01,  2.0318e-01, -1.6011e-01,  8.6397e-01, -2.2691e-02,\n",
       "        -4.4046e-01, -6.2857e-02, -3.9069e-01,  1.0784e-01,  1.1981e-01,\n",
       "         4.9630e-01, -3.8377e-03, -4.2250e-01,  1.0451e+00, -2.6349e-01,\n",
       "        -1.4386e-01,  2.8545e-01,  2.6442e-01,  5.0848e-01, -3.4544e-01,\n",
       "         2.0938e-01,  3.9765e-01, -2.8668e-01, -2.9289e-01,  1.0621e-01,\n",
       "         1.2800e-01,  4.2874e-01, -7.5396e-02,  2.4798e-01,  8.1016e-02,\n",
       "        -2.1906e-02, -2.7718e-01, -7.3758e-02, -5.9758e-02, -1.1127e-01,\n",
       "        -1.2804e-01,  2.8441e-01, -1.5104e-01, -2.0100e-01,  7.6934e-01,\n",
       "         3.0968e-01, -5.9233e-01, -3.4167e-01, -1.5913e-01, -5.3265e-02,\n",
       "         2.1583e-01, -3.5234e-01, -1.2197e-01,  3.0294e-01,  7.3882e-01,\n",
       "         1.3296e-01, -3.4018e-01, -8.1062e-02, -2.5973e-01,  1.7507e-01,\n",
       "         2.3974e-02,  4.1746e-01, -6.2069e+00,  1.1619e-01, -5.7523e-01,\n",
       "        -2.6148e-01, -3.7661e-01, -2.8330e-01,  1.3297e-01,  3.6957e-01,\n",
       "         7.4661e-02, -3.4777e-01,  1.5740e-01,  3.4407e-01, -5.0272e-01,\n",
       "         1.7432e-01,  7.9398e-01,  7.3562e-01], device='cuda:0')"
      ]
     },
     "execution_count": 153,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents[0].embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zX4ZkKaPPIph",
    "outputId": "39bdae6a-d7fc-41c2-db68-a52c773ffd78"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([], device='cuda:0')"
      ]
     },
     "execution_count": 154,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents[0].tokens[31].embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {
    "id": "w8XT_0ahPIph"
   },
   "outputs": [],
   "source": [
    "def get_embeddings(embeddings, sentence):\n",
    "    sent = Sentence(sentence)\n",
    "    embeddings.embed(sent)\n",
    "    if len(sent.embedding):\n",
    "        return sent.embedding.float()\n",
    "    else:\n",
    "        return torch.stack([token.embedding for token in sent.tokens]).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2yt6esWrPIph",
    "outputId": "eb6821d1-b68a-49c5-8abc-2e3266558349"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-6.4245e-02,  3.5365e-01, -2.4962e-01, -5.3912e-01, -1.9917e-01,\n",
       "        -2.7712e-01,  1.6942e-01,  1.0867e-01, -4.0799e-01, -1.4945e-01,\n",
       "         4.4679e-02, -1.9687e-01, -3.0574e-01,  9.6215e-02,  2.8053e-01,\n",
       "         9.0462e-01, -3.0250e-01,  1.7854e-01, -1.6856e-01, -1.7227e-01,\n",
       "        -1.8270e-01,  4.5121e-01,  3.6128e-01,  1.2309e-01, -1.2047e-02,\n",
       "        -5.0255e-01,  4.2645e-01, -3.7183e-02, -1.7052e-01, -4.8920e-02,\n",
       "         1.0503e-01,  3.3456e-01, -3.8483e-02, -4.7287e-01, -7.4431e-02,\n",
       "         2.3990e-01, -8.3231e-02,  1.7974e-01,  4.8939e-01,  9.3028e-03,\n",
       "        -4.2722e-01,  2.6338e-01, -1.3217e-01, -1.3950e-01, -1.1859e-01,\n",
       "         4.5835e-02, -3.9031e+00, -1.0726e-01,  5.1103e-02, -9.9710e-02,\n",
       "         1.5913e-01, -9.5331e-02,  2.1513e-01,  4.7707e-01,  2.1226e-01,\n",
       "         4.3450e-01, -4.5154e-01,  1.8344e-01, -1.0171e-01, -3.6373e-01,\n",
       "         8.9560e-01,  3.5931e-01, -2.8519e-01,  2.9428e-02,  3.3968e-01,\n",
       "         4.1062e-01, -2.1050e-02,  2.7801e-02,  2.8888e-01,  4.5761e-01,\n",
       "        -3.8254e-01, -1.9211e-01, -2.6551e-01,  1.7064e-01, -2.0920e-01,\n",
       "        -3.3608e-01, -1.0044e-01, -2.9611e-01, -5.6573e-01,  4.3841e-01,\n",
       "        -8.9239e-02, -1.4645e-02,  2.2009e-01,  2.5755e-01, -2.6711e-01,\n",
       "        -1.4440e-01, -2.8944e-01,  2.2547e-01,  6.0303e-01,  1.0250e-01,\n",
       "        -2.0606e-01, -1.5392e-01, -3.2992e-03,  7.7604e-02,  1.8428e-01,\n",
       "         1.6114e-01,  2.5668e-01, -8.3616e-02,  2.7078e-01, -5.4946e-02,\n",
       "         4.9633e-01, -2.7734e-01,  4.1486e-01, -5.3347e-01, -7.8131e-01,\n",
       "         1.7047e-01,  2.8687e-01,  4.9673e-02,  4.6922e-01, -1.6500e+00,\n",
       "         4.6991e-01, -1.5328e-01,  2.7729e-01, -1.0157e+00,  1.5022e-01,\n",
       "        -1.7466e-01,  4.1232e-02, -4.4218e-01, -5.0299e-01,  1.1937e-01,\n",
       "         1.8505e-01,  1.7673e-01,  2.1935e-01, -1.3052e+00, -3.4139e-01,\n",
       "        -3.8844e-02, -2.4191e-01,  2.5906e-01,  1.2551e-01, -1.7339e-02,\n",
       "         4.4672e-01,  3.3219e-01, -2.1076e-01, -2.6042e-01,  2.1485e-01,\n",
       "         1.6804e-01, -1.6398e-01,  1.1147e-01, -1.4731e-01,  1.0370e-01,\n",
       "        -2.0025e-01, -1.7985e-01, -2.5822e+00, -9.3411e-02,  5.8352e-01,\n",
       "         4.2851e-01,  1.3719e-01, -4.4423e-01, -2.4912e-03,  6.3892e-01,\n",
       "         4.1029e-01, -3.2187e-01,  5.8879e-02, -2.1344e-01, -4.0472e-02,\n",
       "        -4.0414e-01,  3.8056e-01,  1.7888e-01, -4.1893e-01,  1.6880e-01,\n",
       "        -4.6561e-02, -3.3560e-01,  1.3599e-02,  2.4541e-01, -1.9910e-01,\n",
       "         5.3978e-01,  5.7400e-01,  5.7995e-01, -3.8399e-02, -4.4134e-02,\n",
       "         3.8312e-01, -1.6357e-02,  7.0812e-01, -1.6653e-01, -4.0108e-01,\n",
       "         8.2581e-02,  4.2481e-01, -1.2611e-01,  4.2772e-01, -7.9697e-02,\n",
       "        -7.4779e-01,  7.4372e-02,  2.0130e-01,  8.0436e-01,  8.2707e-01,\n",
       "         3.8594e-01, -2.6690e-02, -2.8838e-01,  2.1367e-01,  2.3883e-01,\n",
       "         1.4772e-01, -1.7737e-01, -1.6458e-01, -4.7632e-01, -9.9866e-02,\n",
       "        -7.7609e-02,  3.6381e-01, -5.1024e-01, -3.6723e-01, -2.4667e-01,\n",
       "        -2.0106e-01,  1.8970e-01,  1.2936e-01, -7.0922e-01,  1.8065e-01,\n",
       "         4.2441e+00,  3.2519e-01, -3.5622e-01,  7.9891e-02,  4.0457e-01,\n",
       "        -5.2027e-01, -4.0209e-01,  1.1543e-01,  5.4013e-01, -8.5186e-01,\n",
       "         5.7481e-01,  3.0116e-01,  3.0603e-01,  3.0003e-02, -3.7785e-01,\n",
       "        -2.3895e-01,  1.0855e-01, -7.3347e-02, -2.2569e-02, -1.8636e-01,\n",
       "         5.5137e-01, -2.5551e-01,  5.6109e-01,  8.9961e-02, -1.4174e+00,\n",
       "         1.9548e-01,  1.8128e-01,  5.7270e-01,  3.7286e-01,  2.1191e-02,\n",
       "         2.0575e-01, -2.7981e-01, -4.0403e-02, -5.4554e-02, -2.4785e-01,\n",
       "        -2.4466e-01, -4.0418e-03,  8.9292e-02,  5.5506e-01, -1.7438e-01,\n",
       "         4.4969e-01,  3.9400e-01, -5.3704e-01,  6.6374e-01,  3.8592e-01,\n",
       "         6.9357e-01,  6.0498e-01, -2.1726e-01, -1.0124e-01, -2.2627e-01,\n",
       "         5.7905e-01,  4.2297e-01, -1.2389e-01,  2.0031e-01,  4.5129e-01,\n",
       "         1.7262e-01,  1.0963e-02,  3.6436e-01,  5.7858e-01, -5.3210e-01,\n",
       "        -1.4754e-02,  6.1678e-01, -3.0254e-01, -3.7331e-01,  4.2933e-02,\n",
       "        -2.8411e-01,  2.5481e-01, -3.1135e-01, -2.4889e+00,  1.2208e-01,\n",
       "        -8.3916e-02, -1.0063e-01,  1.6028e-01,  9.8307e-02, -1.5816e-01,\n",
       "        -3.4994e-02, -1.4911e-01, -8.0463e-01,  5.4030e-01,  7.1562e-02,\n",
       "         1.7398e-01,  1.4491e-01, -7.0773e-01,  3.9191e-01,  1.0117e-01,\n",
       "        -2.8370e-01,  2.8936e-01,  1.5133e-01,  5.0763e-02, -2.4023e-01,\n",
       "         4.4110e-01, -3.9493e-02,  2.6435e-01,  3.1356e-01, -1.8297e-01,\n",
       "        -2.4273e-01, -1.7542e-01,  1.4501e-01,  3.0055e-01, -2.8845e-01,\n",
       "        -1.5901e-01,  1.7493e-01, -2.9559e-01, -3.4490e+00,  1.3099e-01,\n",
       "        -3.8233e-01, -1.0336e-01,  6.8291e-01,  6.9417e-01,  6.1574e-01,\n",
       "        -8.2423e-02, -1.2499e-01, -1.2243e-01,  2.8974e-01, -4.1564e-01,\n",
       "         6.8799e-02,  6.7222e-01,  5.1716e-01, -3.0253e-02,  3.6390e-01,\n",
       "        -5.9201e-01,  2.2436e-01, -2.7201e-01, -8.8383e-02, -5.5831e-01,\n",
       "         1.0460e-01, -3.3686e-01,  4.6233e-01,  6.3347e-01, -5.7720e-01,\n",
       "        -4.1045e-01, -6.3033e-02, -2.6181e-01,  2.5220e-01, -5.7010e-01,\n",
       "        -4.4242e-02, -2.5703e-03, -2.6010e-01, -2.0902e-01,  2.1671e-01,\n",
       "        -1.5338e-01,  2.3135e-02,  8.3758e-02,  2.4785e-01,  8.1829e-01,\n",
       "         4.6562e-01,  1.6161e-01,  1.5305e-01, -2.2000e-01, -2.5913e-01,\n",
       "        -3.0355e-01,  2.0841e-01, -8.6139e-02, -1.0182e-01,  7.8481e-01,\n",
       "         1.0315e+00, -5.8092e-01, -2.5592e-01,  1.4235e-01,  2.2627e-01,\n",
       "        -3.8135e-01, -1.7177e-01,  6.8986e-01,  9.5218e-01, -4.2518e-01,\n",
       "         2.9856e-01, -7.7871e-01,  4.6342e-01,  3.8082e-01,  3.3461e-01,\n",
       "        -3.6359e-01,  1.3449e-01,  6.0971e-02, -6.3682e-01,  6.5791e-01,\n",
       "        -3.4899e-01, -1.6919e+00, -1.1290e-01, -8.9324e-03,  7.0164e-01,\n",
       "        -2.4298e-01, -5.0133e-01, -9.4679e-02,  9.2998e-02, -6.9514e-01,\n",
       "         1.6732e-01,  1.0738e-01, -3.5135e-01, -4.1886e-01, -5.6835e-02,\n",
       "         1.8974e-01, -6.7744e-01, -5.0675e-02, -2.8237e-01, -9.4652e-02,\n",
       "         2.3333e-01,  2.1024e-01, -1.3943e-01,  4.7849e-01, -3.5598e-02,\n",
       "        -5.0288e-02,  2.2644e-01,  1.7870e-01, -8.6450e-02, -4.6139e-01,\n",
       "        -9.7895e-02,  9.5690e-03, -6.0332e-01, -1.0339e-01, -6.9934e-01,\n",
       "         8.6564e-01,  2.4269e-01,  1.2407e-01, -4.4824e-01,  6.2005e-02,\n",
       "         1.1447e-01, -4.3741e-02,  1.5308e+00, -4.7716e-01,  4.9675e-01,\n",
       "         8.2663e-01,  4.4054e-01, -8.6616e-02,  2.1151e-01, -1.6980e-02,\n",
       "         3.6450e-02,  5.2057e-02, -4.3879e-01, -2.0526e-01, -1.0826e-01,\n",
       "        -2.0654e-01,  5.4325e-02, -2.3006e-01, -2.3910e-01,  3.5886e-01,\n",
       "        -3.6865e-01, -4.1345e-01,  3.2852e-01, -2.1309e-01, -7.7967e-01,\n",
       "        -4.0635e-01,  4.1312e-01,  4.5644e-01,  5.6297e-01, -4.0113e-02,\n",
       "        -7.8324e-02,  3.5196e-01, -4.0508e-01,  4.9119e-01,  2.8871e-01,\n",
       "         1.2004e-02,  2.1122e-02,  5.8201e-01,  7.2554e-02, -1.2406e-01,\n",
       "        -2.2747e-01, -3.6875e-01,  1.2822e-01, -1.3220e-02, -2.5432e-01,\n",
       "        -5.4382e-01, -1.6104e-01, -4.9271e-01, -2.6912e-05, -1.6866e-01,\n",
       "        -2.0367e+00, -1.8150e-01,  7.2432e-01,  2.6644e-01,  2.3441e-01,\n",
       "         1.3906e-01, -7.0421e-03,  4.3320e-01, -3.9089e-01,  3.8912e-01,\n",
       "        -1.3505e-01,  1.8608e-01, -2.7714e-01,  7.7001e-02, -3.5178e-01,\n",
       "         3.1395e-01, -6.1126e-01, -1.9384e-01,  3.4077e-02,  7.6535e-02,\n",
       "        -6.3849e-01,  9.1557e-02,  6.1446e-01, -9.3746e-02,  2.7309e-02,\n",
       "        -2.9999e-01,  8.1415e-02,  2.3125e-01,  4.6289e-02,  1.2134e-01,\n",
       "         1.3592e-01, -5.6097e-01, -2.1794e-01, -6.8162e-01,  6.4019e-01,\n",
       "        -1.8568e-01, -1.6626e-01,  3.4131e-01,  1.8143e-01,  1.4511e-01,\n",
       "         4.1716e-02,  3.5290e-01,  2.6542e-01, -3.3044e-01,  6.5737e-01,\n",
       "        -4.8462e-01, -4.2369e-01,  5.4556e-01, -4.1632e-01, -7.4869e-02,\n",
       "        -2.7194e-01, -5.6729e-02, -1.9845e-01, -2.5328e-01, -8.4373e-01,\n",
       "        -2.5077e-01, -7.6245e-02,  1.3954e-01, -7.3810e-01, -7.1694e-02,\n",
       "         3.7180e-01, -6.6641e-01, -7.3948e-02,  1.0741e-02,  6.6874e-02,\n",
       "        -5.4790e-01, -2.3494e-01, -5.4339e-02, -2.7594e-01,  2.3641e-01,\n",
       "         9.0829e-02, -2.4924e-02, -6.5618e-01, -1.5407e-01,  6.5145e-03,\n",
       "        -1.8896e-01, -4.8536e-01,  4.1967e-01,  4.9990e-01, -2.3470e-01,\n",
       "         2.3999e-01,  2.2488e-02,  7.1898e-02,  3.7770e-01,  2.2666e-01,\n",
       "        -4.6463e-02,  7.9462e-02, -1.3600e-01,  2.6624e-01, -1.0557e-01,\n",
       "        -5.1542e-01, -2.1026e-01, -3.6529e-01,  3.7681e-01,  4.2558e-01,\n",
       "        -2.0673e-01,  1.7708e-01, -9.5000e-01,  3.5931e-01, -6.9485e-01,\n",
       "         2.6662e-01, -3.8393e-01, -1.4876e-01,  3.5606e-01, -1.9180e-01,\n",
       "         5.2490e-02,  6.9162e-01,  1.2743e-01, -2.9465e-01, -6.5748e-01,\n",
       "        -1.3319e-01, -1.6700e-01,  3.4043e-01,  1.0655e-01,  1.4711e-01,\n",
       "        -5.7975e-01, -1.9442e-01, -5.5820e-01,  1.5113e+00,  2.5854e-01,\n",
       "        -7.7182e-02,  1.9113e-01, -2.0155e-01,  5.2039e-02, -6.6810e-02,\n",
       "         7.7022e-01,  4.3337e-01,  2.1713e-01, -1.9559e-01,  1.1622e-01,\n",
       "         6.9702e-02,  1.9887e-01,  4.4560e-01, -2.0377e-01,  9.3892e-02,\n",
       "        -3.8858e-01, -1.0287e+00, -8.7258e-02,  3.0796e-01,  1.1579e-01,\n",
       "         2.2186e-01,  2.4396e-02, -2.2758e-01,  7.7906e-01,  8.9298e-02,\n",
       "        -5.7492e-01, -2.8990e-01,  5.5056e-01, -3.1180e-01,  5.1760e-01,\n",
       "        -2.1880e-01,  3.5772e-01, -9.8653e-01,  3.4400e-02, -1.5934e-01,\n",
       "        -2.6761e-01,  4.0133e-02,  7.4575e-02,  3.0935e-01,  2.3991e-02,\n",
       "        -2.4070e-01,  1.9046e-01, -3.6814e-01,  3.0126e-01,  1.0484e-01,\n",
       "         3.0939e-02,  3.2499e-01,  4.0934e-01, -1.7192e-01,  5.4189e-01,\n",
       "         3.5965e-01,  1.4977e-01, -2.0299e-01,  2.5858e-02, -2.3124e-01,\n",
       "        -5.2681e-01, -5.0653e-02, -4.0077e-01,  3.5185e-02,  6.1388e-01,\n",
       "         1.7713e-01, -5.8146e-01,  1.2276e-01,  2.9924e-02, -5.3960e-02,\n",
       "        -4.8978e-01,  8.5425e-02, -1.6308e-01,  2.2882e-02,  2.6261e-01,\n",
       "         2.0515e-01,  3.4557e-01,  2.7707e-01,  3.1900e-01,  6.0790e-01,\n",
       "        -5.5657e-01, -4.6023e-01, -1.5954e+00,  1.5275e-01,  3.5691e-01,\n",
       "         4.9637e-01, -1.7432e-01,  1.8391e-01,  5.2575e-01,  1.6172e-01,\n",
       "        -1.6158e-01,  1.2285e-01,  1.7329e-01,  1.0693e+00,  4.1782e-01,\n",
       "         2.6315e-01, -2.6413e-02, -3.4535e-01,  3.1312e-01,  3.1303e-03,\n",
       "        -1.3553e-01,  9.2502e-02,  1.8213e-01,  1.0432e-01, -5.5064e-02,\n",
       "        -5.2151e-01, -5.9766e-01, -3.0186e-01,  1.1254e-02,  6.2411e-01,\n",
       "        -2.4277e-01,  2.0318e-01, -1.6011e-01,  8.6397e-01, -2.2691e-02,\n",
       "        -4.4046e-01, -6.2857e-02, -3.9069e-01,  1.0784e-01,  1.1981e-01,\n",
       "         4.9630e-01, -3.8376e-03, -4.2250e-01,  1.0451e+00, -2.6349e-01,\n",
       "        -1.4386e-01,  2.8545e-01,  2.6442e-01,  5.0848e-01, -3.4544e-01,\n",
       "         2.0938e-01,  3.9765e-01, -2.8668e-01, -2.9289e-01,  1.0621e-01,\n",
       "         1.2800e-01,  4.2874e-01, -7.5396e-02,  2.4798e-01,  8.1016e-02,\n",
       "        -2.1906e-02, -2.7718e-01, -7.3758e-02, -5.9758e-02, -1.1127e-01,\n",
       "        -1.2804e-01,  2.8441e-01, -1.5104e-01, -2.0100e-01,  7.6934e-01,\n",
       "         3.0968e-01, -5.9233e-01, -3.4167e-01, -1.5913e-01, -5.3265e-02,\n",
       "         2.1583e-01, -3.5234e-01, -1.2197e-01,  3.0294e-01,  7.3882e-01,\n",
       "         1.3297e-01, -3.4018e-01, -8.1062e-02, -2.5973e-01,  1.7507e-01,\n",
       "         2.3974e-02,  4.1746e-01, -6.2069e+00,  1.1619e-01, -5.7523e-01,\n",
       "        -2.6148e-01, -3.7661e-01, -2.8330e-01,  1.3297e-01,  3.6957e-01,\n",
       "         7.4661e-02, -3.4777e-01,  1.5740e-01,  3.4407e-01, -5.0272e-01,\n",
       "         1.7432e-01,  7.9398e-01,  7.3562e-01], device='cuda:0')"
      ]
     },
     "execution_count": 156,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_embeddings(bert_doc, watch1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dp3LpDtwPIph"
   },
   "source": [
    "## Model III - Preprocessing Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xafyHRinPIpi"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "134448f816dd4e1c9fc331e98d7381b9",
      "a0d6a4f644164ad5b8886e50f37fdee7",
      "ae459920284243408cf298063ea85720",
      "f6bfeefba2ae41a786d03af7c2875bea",
      "ae0ece405c544406b9e0c90a665d068c",
      "aed9c03c8e9d4e0f8eda236bd013f637",
      "e01d1f7a9092482f8b03bee5770116bb",
      "89ca84846175417eb38fd7f405a5da93",
      "25a7341afc894965bc384be1379e07dd",
      "0a799174580945b5a21137b528c5da56",
      "d0f6ba92d7484d9aa9ddc84f5318ccba",
      "162b098de96a4b2084685c0a77a0eacb",
      "e47034499cc34e38bc0caba6d32e0af2",
      "f12561a7885745f68335b541d6e33baa",
      "a229f1dee3d043d49b4f337d21aee795",
      "b9f77f25becb47958456c513715fa705",
      "15e6c1e653ea43ceac91c1188d2d9b4b",
      "1d110c4fecb74297b1d648948a410fcd",
      "e6fc878e9bd94c44a1121ed9fb94af18",
      "81bb103bdcff45b4955023635434a513",
      "e413a8f688144c74ba4028bf4ed854d5",
      "1177ccace23c41cd9d4a64ac9a214106"
     ]
    },
    "id": "62H2IcGYPIpi",
    "outputId": "90faa44b-5eaa-496f-de1b-de28c97eb715"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "134448f816dd4e1c9fc331e98d7381b9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "162b098de96a4b2084685c0a77a0eacb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_dataset_doc = train_dataset.map(lambda row: {'embeddings': get_embeddings(bert_doc, row['sentence'])})\n",
    "test_dataset_doc = test_dataset.map(lambda row: {'embeddings': get_embeddings(bert_doc, row['sentence'])})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {
    "id": "q5rURvw6PIpi"
   },
   "outputs": [],
   "source": [
    "train_dataset_doc.set_format(type='torch', columns=['embeddings', 'labels'])\n",
    "test_dataset_doc.set_format(type='torch', columns=['embeddings', 'labels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "t9Ov7ucKPIpi",
    "outputId": "fed21982-e665-4e43-caf4-2e432cd29798"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.0958,  0.2077, -0.5529,  ...,  0.3126,  0.1283,  0.3483],\n",
       "        [ 0.0290,  0.6617,  0.3451,  ..., -0.0209,  0.2655,  0.6687],\n",
       "        [-0.0159,  0.0439, -0.1976,  ...,  0.1954,  0.5707,  0.8137],\n",
       "        ...,\n",
       "        [ 0.0975, -0.0047, -0.5183,  ...,  0.5229,  0.7665,  0.2394],\n",
       "        [ 0.2676,  0.5103, -0.0146,  ...,  0.2528,  0.5094,  0.8005],\n",
       "        [ 0.3020,  0.2819, -0.2024,  ..., -0.0471,  0.4129,  0.5016]])"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dataset_doc['embeddings']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "id": "OfMQkFCHPIpi"
   },
   "outputs": [],
   "source": [
    "train_dataset_doc = TensorDataset(train_dataset_doc['embeddings'].float(),\n",
    "                                  train_dataset_doc['labels'].view(-1, 1).float())\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_dataset_doc, batch_size=32, shuffle=True, generator=generator)\n",
    "\n",
    "test_dataset_doc = TensorDataset(test_dataset_doc['embeddings'].float(),\n",
    "                                 test_dataset_doc['labels'].view(-1, 1).float())\n",
    "test_loader = DataLoader(test_dataset_doc, batch_size=32, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RDOr-sgLPIpi"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {
    "id": "NEjwk7i8PIpj"
   },
   "outputs": [],
   "source": [
    "torch.manual_seed(41)\n",
    "model = nn.Sequential(\n",
    "    # Classifier\n",
    "    nn.Linear(bert_doc.embedding_length, 3),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(3, 1)\n",
    ")\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "id": "93pg9nrOPIpj"
   },
   "outputs": [],
   "source": [
    "sbs_doc_emb = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_doc_emb.set_loaders(train_loader, test_loader)\n",
    "sbs_doc_emb.train(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "5jSTsMaSPIpj",
    "outputId": "bf88835b-a1d4-4040-862a-584617a0c990"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAF6CAYAAAAedNt3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACeAElEQVR4nOzdd3hUxeLG8e/Zlh5CDS30ZqT33hUEQlcU7BVsKD+Fe73Wq1e8114QGxYUKRaUIijSSwDpIr33KiE9287vj5VgTMD0bML7eR4eklPmzMmwS96dOTNGXFyciYiIiIiIiIj8LUtRV0BERERERESkuFCIFhEREREREckmhWgRERERERGRbFKIFhEREREREckmhWgRERERERGRbFKIFhEREREREckmhWgRERERERGRbFKIFhEREREREckmhWgRERERERGRbFKIFhEREREREckmhWgRERERERGRbFKIFr+RmprKvn37SE1NLeqqyB/UJv5F7eF/1Cb+R23if9Qm/kXt4X/UJsWPQrT4FY/HU9RVkL9Qm/gXtYf/UZv4H7WJ/1Gb+Be1h/9RmxQvCtEiIiIiIiIi2aQQLSIiIiIiIpJNCtEiIiIiIiIi2aQQLSIiIiIiIpJNCtEiIiIiIiIi2aQQLSIiIiIiIpJNtqKugIiIiIiIXOT1eklKSiqQdYO9Xi8Oh4Pz58+TkJCQ7+VLzqlNClZgYCAhISFYLPnXf6wQLSIiIiLiJ7xeL2fPniU0NJRy5cphGEa+l+90OnE4HPkaKiT31CYFxzRNUlNTOXv2LGXLls23n69aSfyKacKXXwayZYv+aYqIiMiVJykpidDQUIKCgvI9QItcaQzDICgoiNDQUJKSkvKtXCUV8RtxcQZPPFGLMWNKcffdwSQnF3WNRERERApXamoqgYGBRV0NkRIlMDAwXx+PUIgWv7BqlZUePcry889lANi1y8q//qX/QEREROTKox5okfyV368phWgpcosX2+jXL4SjR60Ztn/ySQCzZ+uxfRERERER8R8K0VLkOnRw06SJJ8t9Dz8cxLFj+jRWRERERET8g0K0FDmHAz76KIXgYG+mfefOWbjvvmA8WWdsEREREZE8iYiIoG/fvnkqY/ny5URERDB+/Ph8qpX4M4Vo8Qu1a3t58cWs18VbvtzG2287CrlGIiIiIlJYIiIicvRH/l6jRo2IjIws6mqUSHrgVPzGsGGpzJ3rYcGCMpn2vfBCIJ07e2jeXF3SIiIiIiXNuHHjMm2bOHEi8fHxWe7LT2vXriUoKChPZbRo0YK1a9dStmzZfKqV+DOFaPEbhgH//OdBtm0rlWmSMbfb4O67g1i6NJGwsCKqoIiIiIgUiH/+85+Ztn355ZfEx8dnuS8/1atXL89lBAcH50s5UjxoOLf4lbAwD+++ex6Lxcy0b98+K+PG5e1TQhEREREpvg4ePEhERASjRo1i586djBgxgpo1axIREcHBgwcBmD17NnfddRfNmjWjUqVKVKtWjeuuu47vv/8+yzKzeiZ61KhRREREcODAAd577z1atWpFhQoVaNiwIS+99BJeb8a5fC71THSjRo1o1KgRiYmJjBs3jgYNGlChQgXat29/yfocPHiQO+64gxo1alClShX69OnDypUrGT9+PBERESxfvjy3P75LSkpK4sUXX6RVq1ZERkZSo0YNbrjhBlavXp3p2NTUVN5++206dOhAtWrVqFy5Mo0aNeL222/n119/TT/O6/UyefJkunfvTo0aNahYsSLR0dEMGzasQO6hMKknWvxOmzYuHn88jf/+N/M60V9+6aBHDzdDhriKoGYiIiIi4g/279/PNddcQ3R0NMOHD+f333/H4fDNofPvf/8bu91O27ZtqVixImfOnGHevHncdttt/Pe//+W+++7L9nWefvppVq5cSa9evejevTtz587lpZdewuVy8dRTT2WrDLfbzeDBg4mLiyMmJoaUlBS+/fZbbr/9dr755hu6du2afuyxY8fo1asXJ06coGfPnjRu3Jjdu3czaNAgOnfunKOfUXalpqbSv39/1q9fT5MmTRg1ahSnTp1i5syZLFy4kEmTJjFw4MD040eNGsXMmTO5+uqrGT58OAEBARw9epTly5ezceNGGjVqBMBzzz3Hm2++Sc2aNbn++usJDQ3l2LFjrF69mqVLl9KpU6cCuZ/CoBAtfunxx9NYssTGmjWZ/4k++mgQLVu6qV49c2+1iIiISEl2zTUheTrfNME0vRiGBSOfVxFdsCApfwu8jNWrVzN27FieeOKJTPu++uoratSokWFbYmIi1157Lf/5z3+45ZZbCA4OztZ1Nm/ezMqVK6lYsSIAY8eOpXnz5nzwwQeMGzcuPbhfzvHjx2nWrBlz5sxJP/76669nwIABTJgwIUOIfvbZZzlx4gRPPfUU//d//5e+/fPPP+ehhx7KVp1z6s0332T9+vXccMMNvP/++xh//MO47777uOaaaxg9ejQ9evQgLCyM8+fP891339G0aVMWLlyI1XrxEUyPx0NCwsWJgidPnkylSpVYuXJlpp/3uXPnCuReCotCtPglmw3efz+Zzp3DiI/P+A4fH29w333BzJmThE3/gkVEROQK8ssv+uUHIDIyksceeyzLfX8N0AChoaEMHz6cJ598kg0bNtCxY8dsXefxxx9PD9AAZcuWpU+fPkydOpXdu3dz9dVXZ6ucF198MUPg7tKlC1FRUWzYsCF9W1paGt9//z3ly5fnwQcfzHD+zTffzFtvvcXu3buzdb2cmDp1Kna7nWeeeSY9QAM0adKEm266ic8++4y5c+dy4403YhgGpmkSGBiIxZLxyWCr1Zpp5nS73Z4haF9QunTpfL+PwqRnosVv1ahh8tprKVnuW73axiuvBBRyjURERETEHzRs2PCSvcCnT5/miSeeoHXr1lSqVCl9Wawnn3wSgBMnTmT7Ok2bNs20rUqVKgCcP38+W2WUKlUqy2BfpUqVDGXs2bOHtLQ0mjVrRkBAxt9zDcOgdevW2a53dsXHx3PgwAFq1aqVfl9/dmHI9YVnncPDw7n22mtZvXo1nTt35tVXX2XNmjW4XJkftRwyZAiHDh2iXbt2vPDCCyxdupSUlKx/ty9uFKLFrw0d6uLGG51Z7vvf/wJYvTrzJ1siIiIiUrKVL18+y+3nzp2jW7duvPvuu5QpU4abb76Zxx57jHHjxtGnTx/A1+ObXWFZLAtzoWfV48ne0qvh4eFZbrdarRkmKLswFLpcuXJZHl+hQoVsXS8nLlzzUj/PC+tM/3mY9qeffspjjz1GfHw8zz//PL169aJ27dr84x//IDk5Of24l156ieeffx6Hw8Err7zCgAEDqFmzJiNHjuTs2bP5fi+FSSFa/N7LL6dQo0bmNymv1+Cee4LJ5oeAIiIiIlJCGJd4oPvzzz/nyJEj/Otf/2L+/Pm8/PLLPPnkk/zzn/+kVatWhVzLnLkQ2M+cOZPl/lOnThXYNU+fPn3Za/75w4Tg4GCefPJJNm/ezKZNm3j77bepU6cO7733XoZn1G02Gw899BCrV69m+/btfPTRR7Rr145p06Zxzz335Pu9FCY9VCF+LywMJk1KoVevENzujG+Yhw9bGDMmiI8+Ssn3yTFERERE/E2rVu48nV+QE4v5g/379wOk9zr/WWxsbGFXJ0fq1KlDQEAAmzZtIi0tLcOQbtM0+eWXX/L9muHh4dSoUYN9+/Zx7NgxKleunGH/ihUrANJn3P6rGjVqUKNGDYYOHUrdunWZN28eb7zxRqbjKlWqxNChQxk8eDAtW7ZkyZIlpKSkEBRUPJevVYiWYqFFCw9PPJHGv/+dedmrb75x0LOnm5tu0rJXIiIiUrLldQZsr9eL0+nE4XBkmhiqJIiKigJ8s3f/edKvr776ip9++qmoqpUtAQEBDBgwgBkzZjBx4kQeeeSR9H1Tp05l165dBXLdm266ifHjx/Pcc8/x3nvvpffyb926lS+//JLw8PD0dbTPnDnDqVOniI6OzlBGXFwcaWlplClTBvANmd+0aRNt2rTJcFxSUhJJSUnY7fZi/e9PIVqKjdGj01i0yMaKFZn/2T7+eBBt2nioVcubxZkiIiIiciUYNmwYb7zxBmPHjmX58uVERUWxdetWli5dSkxMDLNnzy7qKl7W008/zZIlS3j22WdZuXJl+jrRP/74Iz179uTnn3/OUfh0uVyMGjXqkvsnTpzI6NGj+emnn5g+fTq7du2iS5cunD59mpkzZ+J2u3n//ffTh3MfO3aMzp0707BhQ66++moqV67M77//zg8//IDL5UpfhislJYVevXpRp04dmjZtStWqVUlKSmL+/PmcPHmShx56KNPkacWJQrQUG1arb9mrDh1CiYvL+OaRmGhw991B/PhjEnZ7EVVQRERERIpUlSpVmDt3Ls888wxLlizB4/HQuHFjZs6cyZEjR/w+RFetWpWffvqJZ599lkWLFrFy5UqaNGnCt99+y3fffQdkPdnZpXi9XqZOnXrJ/RMnTiQwMJBZs2bxxhtvMHPmTN59912CgoLo0KEDY8aMoV27dunHV6tWjX/84x8sW7aMpUuX8vvvv1O2bFmaNGnCyJEj6dmzJwAhISE899xzLF26lNjYWE6fPk1ERAR16tThmWeeYciQIbn7AfkJIy4uzizqSogApKamcvjwYaKioggMzDxs+4JZs2zcemtIlvvGjEnl6aezP+OiXF5220QKh9rD/6hN/I/axP+oTXLm9OnTl5wpOT+U9OHcxVF226R3796sXbuWQ4cOERoaWog1LBny87WlV44UO/37u7nttqyXvXr99QCWLdOyVyIiIiJSPGW1jvX06dNZvXo1Xbt2VYD2AxrOLcXSiy+msGqVld27MwZm0zQYOTKYFSsSKVNGgyxEREREpHhp164djRs3pn79+litVn799VdWrFhBWFgYzz//fFFXT1BPtBRTISHw0UfJ2O2Zg/KxYxYefjgIUxlaRERERIqZO++8k9OnTzNt2jQ+/PBD9uzZw/XXX8/ChQszzDguRUc90VJsNWni5ZlnUnnyyczry82ZY2fyZDu33aZlr0RERESk+Hjqqad46qmniroachnqiZZi7f77nXTvnnVQ/sc/gti1S//ERUREREQk/yhhSLFmscDEiSmUK5d5feiUFIO77gomTZN1i4iIiIhIPlGIlmIvMtJkwoSULPf9+quVf/9by2mIiIiIiEj+UIiWEqFXLzf33pt1l/OECQEsXKjH/0VEREREJO8UoqXE+Pe/U4mO9mS5b9SoIE6fNgq5RiIiIiIiUtIoREuJERjoW/YqICDz2lanTll44AEteyUiIiIiInmjEC0lSnS0lxdeSM1y308/2fngA0ch10hEREREREoShWjxK5UmTSJ89Ghs8+eT22m1777bSa9eWS979fTTgWzdqn/2IiIiIiKSO0oT4j9Mk3KzZhE0fTohN95IeJ06BN19N7bvv4ekpGwXYxgwYUIKkZGZl71KSzO4++5gUrKezFtEREREROSyFKLFb9i2biXg2LH0742EBBxff03IbbcRXqcOwTffjH36dDh//m/LKlfO5L33sk7KO3ZYefppLXslIiIiciWZMmUKERERTJkyJcP2Ro0a0ahRozyXk5/Gjx9PREQEy5cvL7BrSO4pRIvfCJg795L7jJQU7HPmEHzffb5Aff312CdPxjhz5pLndOvm5qGHsh4S/uGHAcybp2WvRERERPzB3XffTUREBF9//fVlj4uPj6dSpUpUq1aNlGI8tHD58uVEREQwfvz4oq5KtlwI9d98801RV8UvKESL3wi8TIj+M8Plwr5gAcEPP0xYvXqExMTg+OADjD/1Yl/w1FOpNGmS9bJXDzwQxPHjWvZKREREpKjdcsstAHzxxReXPe6bb74hJSWFoUOHEhQUlC/XnjVrFrNmzcqXsvLLvffey9q1a2nRokVRV0WyoBAt/iEtDWfHjjjLlcvRaYbXi235coLGjiU8OpqQa6/F8fbbGAcOAOBw+Ja9Cg7OvLbV779bGDUqCG/mR6dFREREpBB17tyZ6tWrs2zZMg4fPnzJ4y6E7AuhOz/UrFmTmjVr5lt5+aFs2bLUq1eP4ODgoq6KZEEhWvxDQAAJ48ezZe5cfp8zh7QHH8RbrVqOi7GtXUvQU08R3rQpoZ07E/Dyy9T3buell7Ie7rNkiZ0JE7TslYiIiEhRMgyDESNG4PV6L/ms8fbt21m/fj1XX301zZo14/z587zxxhv06dOHBg0aUL58eRo0aMB9993H/v37s33tSz0Tfe7cOR599FHq1q1LpUqV6NatG7Nnz75kOZ9//jk33XQTjRo1IjIykho1ajB48GCWLVuW4bjx48cTExMDwH//+1/KlClDxYoVKVOmDAcPHkw/5lLPRM+bN49+/fpRrVo1KlasSIcOHXjnnXdwu90Zjjt48CARERGMGjWKffv2MWLECKpXr07lypUZMGAAv/76a7Z/RjmV3ToCLFu2jKFDh9KgQQMqVKhA3bp1ue666/j0008zHLdp0yZuvfVWGjZsSIUKFahduzbdunXjlVdeKbD7uBQ9FCr+xWLB1bIlqR07kvr881i2bME+ezb2WbOw7tqVo6KsW7Zg3bKFwP/8h/vr1yey7iDG776BTTQFLg7j/ve/A+nUyU3TpuqSFhEREf8Wcs01eSvANPGaJhbD8C1pko+SFizI0/nDhw/npZde4ssvv2TcuHEYf6nfhXB9oRd6165dvPjii3Tq1Il+/foRHBzMrl27+Prrr/npp59YunQp1XLRKQOQnJxM37592bZtG61bt6ZDhw4cPXqUO++8k+7du2d5zuOPP07Dhg3p2rUr5cqV49ixY/zwww8MHDiQzz//nL59+wLQsWNHDh06xNSpU+nQoQMdOnTA4/FgtVopVarUZev1zjvv8OSTT1K6dGmGDh1KcHAw8+bN48knnyQ2NpYvvvgi08/t0KFD9OzZkwYNGnDzzTezf/9+fvjhB2JiYli7di0VKlTI1c8oP+r4448/cuONN1KqVCn69OlDxYoVOXPmDFu3bmX69OncfvvtAGzZsoVevXphtVrp06cPUVFRnD9/nh07dvDpp5/y2GOP5es9/B2F6BJg1qxZfPzxx2zatIm4uDg2b95M9erVi7paeWcYeJs0Ia1JE9KefBLLzp3YZ83yBeocfnJm3bmTG3iJG3iJfdTkWwbzDUNYQxtcLgt33RXM0qWJhIYW0L2IiIiI5APbL78UdRUKTNWqVenevTs///wzy5Yto0uXLun73G43M2bMICAggGHDhgFQr149du7cSenSpTOUs2zZMgYOHMgrr7zCW2+9lau6vPnmm2zbto3bbruNN998M337sGHDGDJkSJbnrF69mho1amTYduLECbp168bTTz+dHqI7deoEwNSpU+nYsSPjxo3D6XTicDiwWC49UHj//v08++yzlC9fnsWLF1O1alUAnnrqKQYOHMjcuXOZPn06N954Y4bzVq5cybPPPssjjzySvu2FF17glVdeYcqUKTz66KPZ/rn8nZzW8YsvvsA0TWbPnp1pNMDvv/+e/vX06dNJS0tjypQp6T/HrI4rLBrOXQIkJyfTvn17nnjiiaKuSoHy1q9P2uOPk7h8OfGbNpHy/PO4W7XKcTm12M9jvEos7TlMFG/zIFF7l/GvcfYCqLWIiIiIZNelJhibP38+p06dok+fPumhuVSpUpkCNPier27QoAFLlizJdT2mTZuGw+HI9Pt1jx49MoT7P/trgAaoWLEiMTEx7N27l0OHDuW6PgBfffUVbrebBx98MD2cAgQEBPDss88C8OWXX2Y6r3r16jz88MMZtl34OW/YsCFPdcqvOmY1SVyZMmVyfVxBU4guAW688UbGjh1Lx44di7oqhcasUQPnQw+RtGAB8du2kfK//+Hu2BHzMp/eZaUKx3iQCSymOy9Pqc6p/qOxLVgATmcB1VxERERELqVPnz6UK1eOOXPmcP78+fTtl5pQbPny5QwfPpz69etTrlw5IiIiiIiIYNu2bZw4cSJXdYiPj+fgwYPUqlWLyMjITPvbtWuX5XkHDhzg4YcfpmnTpkRGRqbX5YMPPgDIdX0u2LJlC0CWv/O3bt2awMDALJ9zbtSoUaYe7ipVqgBk+Bnnh5zW8UKvfs+ePXn88ceZPXs2Z8+ezXTuoEGDsFgs3HzzzTzwwAN8/fXXHMtiZZ7CUmxD9BtvvJH+D/OXIhjWMn36dB555BG6du1KhQoVsrXg+oYNG7j++uupVq0alStXpmfPnsycObOQalxymZUr47z3XpLmzCFh506S33oLV8+emPac9SyX5wx1l31GyPXXE16nDkH33ott9mxITi6gmouIiIjIn9ntdoYNG0ZKSkr6mtEnT57k559/pmrVqnTt2jX92O+++47+/fuzfPly2rZty6hRoxg7dizjxo0jKioKZy47RRISEgAod4lVY7J6hnjfvn107dqVKVOmUKNGDe644w4ef/xxxo0bR4cOHQBIS0vLVX3+Wq/y5ctn2mcYBuXLl08/5s/CwsIybbPZfE/1ejxZLwVbWHUcOHAgU6ZMITo6mo8//phbbrmFOnXq0L9///RADtCyZUvmzJlD+/bt+frrr7n77ruJjo6me/fumSZuKwzF8pnobdu2MX78eEJCQkhKSiqSOrzwwgscPnyYsmXLEhkZedmp+MH3bMaQIUMIDAxk8ODBhIaGMmvWLO644w6OHDnCQw89VEg1L9nM8uVx3Xorrltvhbg47D/+iH3WLGwLF2Kkpma7HCM+HseMGThmzMAMDsbdsyeu/v1xXXsthIcX4B2IiIiIXFpuHmXLoAAnFssvt9xyCxMmTODzzz/nrrvuYvr06bjdbkaMGJGhR/Wll14iMDCQJUuWULt27QxlfPvtt7m+/oXQeebMmSz3nzp1KtO2d999l7i4ON5///30Z7YvePTRR1m5cmWu6/PXep0+fTrThGmmaXL69OksA3Nhyk0d+/btS9++fUlISGDNmjXMnj2bzz//nKFDh7J27VoiIiIAaN++Pe3btyclJYV169Yxf/58Jk2axLBhw4iNjc1yOH1BKXYh2uVyMWrUKBo1akStWrWYMWNGts+dNm0a7du3v+QsfR6Ph4kTJ3LvvfficFx+2aO3336bWrVqUa1aNV5//XWee+65Sx7rdrsZPXo0FouFuXPn0rhxYwDGjh1Ljx49eP755xkwYECGej377LO88cYbl61DXFzcZfdf8SIicA0bhmvYMEhKwvbzz76JyX78ESMxMdvFGMnJ6ROamQEBOO+7j9R//QsCAgqw8iIiIiKZ5XUGbK/Xm61JrIpSgwYNaNWqFb/88gtbt25lypQp6Utg/dn+/ftp0KBBpgB94sQJDhw4kOvrh4eHU716dfbt28fJkyczDemOjY3NdM6FJbX69OmTYbtpmqxZsybT8VarFchZT3Djxo2ZM2cOK1asoEWLFhn2rVu3jtTUVFq3bp3t8gpCXuoYFhZGz5496dmzJx6Phy+++IL169fTo0ePDMcFBQXRqVMnOnXqRKlSpXjxxRdZvHgxd9xxR4Hd11/55yvnMl555RV27NjBO++8k/6PLzuOHj3K6NGj6devX5YP9Xu9XkaNGsWTTz6ZaU2yrHTt2jXbU+YvW7aM/fv3M3To0PQADb7JEMaMGYPT6WTq1KkZznnwwQdZu3btZf9IDoSE4B4wgJRJk4jfs4ekadNwDh9OoiPzZBSXY6SlEfDWWwQPGwY5COIiIiIikn0Xnn1+7LHH2LlzZ5a/e0dFRbF///4MPcOpqamMGTMGl8uVp+sPGzYMp9PJiy++mGH7okWLWLp0aabjo6KiAN8M3X/2+uuvs23btkzHX5gQ7ejRo9mu0/XXX4/NZmPChAkcP348fbvT6eSZZ54BfMuEFaWc1nHlypVZfpBw+vRpwDchGcDatWtJzWJU6V+PKyzFqid606ZNvPrqqzzxxBM0aNAgR+dWqVKFjz76iDvuuIOYmBjmzJmT/o/d6/UycuRIZsyYwY033sjdd9+dr/VesWIFQJZryl34ZOWvQzzKlSt3yecwJI8CA3H37o27d2/O/9vFfe030PHUTAYxk4qczFYR9iVLCBk8mKQZM+CPISYiIiIikj8GDRrEP//5z/RQ+tcJxQDuvfdexo4dS+fOnenfvz8ej4fFixdjmiYNGzZk69atub7+6NGjmTNnDp999hk7duygffv2HD16lJkzZ9KrVy9+/PHHDMffcccdTJkyhVtvvZWBAwdSpkwZ1q1bx+bNm7M8vl69elSqVIlvv/0Wh8NBZGQkNpuN++6775JrRdesWZNnn32WJ598kg4dOjBo0CCCg4OZP38+u3fvpk+fPpmGkue3SZMm8fPPP2e579Zbb6Vdu3Y5quO4ceM4ceIEbdu2pVq1ahiGwerVq1m/fj2tWrVKn8TtjTfeYMWKFbRr147q1asTGBjI5s2bWbp0KTVq1KBfv34Fet9/VWxCdFpaWvow7tGjR+eqjJiYGCZNmsRdd91Fv379mDNnDlWqVGHUqFHMmDGDG264gXfffTffh7bs3bsXINNQE4DIyEhCQ0PZt29frss/d+4chw8fTh9GsnPnTs6fP09UVFSW0/7LRaXK2bn9i/Zcd901POh5h3bEMoRvGMy3VOfyyxDY1q4ltG9fkmbOxMznRepFRERErmRhYWHpk06VLl0609rAAPfccw92u50PPviAyZMnU6pUKa699lqeeeYZbrvttjxdPyQkhLlz5/Lcc88xZ84cNm/eTIMGDfj444+Jj4/PFIqbNGnCt99+y3/+8x/mzJmDxWKhTZs2zJ8/n3nz5mU63mq18vnnn/PMM8/wzTffkPjHCMdhw4ZdMkSDb7RqrVq1mDBhAjNmzMDpdFK7dm1eeOEFRo4ciVHAz7mvWrWKVatWZbmvY8eOtGvXLkd1HDNmDLNnz2bTpk0sWrQIm81GtWrVeO6557jrrrvSRx7fddddhIeHs379elatWoVpmlStWpX/+7//4/777ye8kOcsMuLi4sxCvWIuPfPMM0ycOJElS5YQHR0NwKhRo5g6dSoLFiygVQ4mWfj222+55557iIqKolmzZsycOZMhQ4bwwQcf5GiI+AUXnomeMGFCpmc1wPdJ2uLFi9mwYQO1atXKtP+qq64iKSkp12vHTZkyhQceeCDT9kvVx1+lpqZy+PBhoqKiCAwMLNRr/+9/Abz44p+vadKC9QzhG0ZV+JqIU3suea6ndm2SvvsO84+RDSVJUbaJZKb28D9qE/+jNvE/apOcOX36dJYzG+eX4vBM9JVGbVI48vO1VSxaae3atbz99ts89thj6QE6LwYPHszEiRM5cOAAM2fOpG/fvrkO0P5gxIgRxMXFZfpTnAJ0Ufu//0ujXTv3n7YYrKclTzCeask72f3ZAryX6G227t1L6HXXYdm9u3AqKyIiIiIiRcbvQ7Tb7WbUqFFcffXVPProo/lSpmmaGdYT27FjR5ZT1eeXC8ML4uPjs9yfkJBQ6EMQJCOrFT74IJnw8MwDMxISLQx/uytxs+bhrVo1y/MtR44Qct11WP60np2IiIiIiJQ8fh+iExMT2bt3L7/++ivly5cnIiIi/c+FGa2vueYaIiIimDNnzt+WZ5omo0eP5osvvmDw4MF88MEHHDhwgJiYmAwzyOWnC89CX3g2+s9OnjxJYmJilsO8pXBFRZm8+WZKlvvWrbMx/utoEufPx1O3bpbHWM6cIbRfP6xZLGMgIiIiIiIlg99PLBYQEJDlbHzge7B97969XHfddZQrV+5vl5wyTZNHHnmEyZMnM2jQID788EOsVisWi4X77rsvfdbuihUr5us9dOjQgddee41FixYxZMiQDPsWLlyYfowUvUGDXCxc6OSLLzKvE/7aawF061adjj/8QMjgwVh//TXTMUZ8PCGDBpE8ZQrubt0Ko8oiIiIiIlKI/D5EBwUF8fbbb2e5b9SoUezdu5cxY8b87cRipmkyZswYPvvsswwBGmDo0KEAGYL0XxdVz4suXbpQo0YNvv76a+677770taLPnz/Pa6+9hsPh4MYbb8y360nevPRSCrGxVvbuzfiMvNdrcN99waxYUR5z9mxCbrwR21/WAgQwkpMJHjaM5I8+wt2/f2FVW0RERERECoHfh+j8cvz4cebMmcPAgQP58MMPsdky3vrQoUMxTZORI0eyZMmSv11jbfLkycTGxgKkL6D++eefp68J3a5dO2699VYAbDYbb731FkOGDKFv374MHjyY0NBQZs2axeHDh3n++eepXr16ft+y5FJoKEyalMw114TicmVcJuDIEQujRwfz6aeQ9M03BN9yC/ZFizKVYTidBN9+OynvvIOriBe9FxERERGR/HPFhOjKlSuzYMECqlatmilAX3D99dfTrFkz6tSp87flxcbGpj+TfcHq1avTF4QH0kM0QOfOnZk/fz7jx49n5syZuFwuoqOjee655xg8eHAu70oKStOmXp56KpWnnw7KtO/77+306RPCyy9baDh1KsH33IN91qxMxxleL8H3309KfDzOkSMLo9oiIiIiIlLAis060VLy+ds6kl4vDBoUwtKlWX/oYrWa3HOPk38+nkjFpx7G8eWXlywr9YknSHv8cTCMSx7jj/ytTa50ag//ozbxP2oT/6M2yRmtE33lUZsUjitunWiRomCxwHvvJVOmjDfL/R6PwXvvBdCqXWk+6fg+aZfpbQ588UUCn3oKTH1mJSIiIpdn6vcFkXyV368phWiRy6hUyeTdd1OwWi/9wjt1ysLI+0PptuktDt3xj0seF/DOOwQ9/DB4PAVRVRERESkBAgMDSU1NLepqiJQoqamp+ToSRiFa5G/07u1m+vRkqlbNukf6gtjVdmp+9iJftfvfJY9xfP45QXffDU5nfldTRERESoCQkBASExNJSUlRj7RIHpmmSUpKComJiYSEhORbuVfMxGIiedGzp5s1axJ47bUA3norINOs3Rd4vQY3xD7Ow6GleT3pPixm5uDtmDkTIzGR5M8+g+Dggq66iIiIFCMWi4WyZcuSlJTEmTNn8r18r9eb3iun52/9g9qkYAUGBlK2bNl8/dkqRItkU0gIPPVUGjfd5GLs2EAWLbJf8ti3Eu/mGKX40hiB3XRl2m9fsICQIUNImjYNSpUqyGqLiIhIMWOxWAgLCyMsLCzfy05NTSU+Pp7IyEhN9OYn1CbFjz7qEMmhOnW8fPNNMpMnJ112iPfXXE+MOYtkMi+TBWCLjSW0f3+Ms2cLqqoiIiIiIpLPFKJFcsEwoH9/N2vXJvDYY6k4HFk/s/QjvenFj5wnPMv91s2bCenTB+PYsYKsroiIiIiI5BOFaJE8CA6GJ59MIzY2kZ49Mw/bBlhBJ7qxmNOUy3K/dedOQnv3xrJ/f0FWVURERERE8oFCtEg+qF3by1dfJfPFF1kP8d5IczqxnCNUyfJ8y6FDhPTujWXbtoKuqoiIiIiI5IFCtEg+MQzo1+/SQ7x30oCOrGAPtbM833LyJCF9+mBdv74wqisiIiIiIrmgEC2Szy4M8V69OpFrrsk4xPsgNejEcn6lYZbnWuLiCBkwAOuyZYVRVRERERERySGFaJECUquWlxkzkpkyJYmoqItDvE9QiS4sZQ2tszzPSEwk5Prrsc2bV1hVFRERERGRbFKIFilAhgF9+7pZsyaBxx9PJSDAN8T7HGXoyc8spHvW56WlEXzzzdi/+qowqysiIiIiIn9DIVqkEAQHw7/+5Rvife21viHeiYTRl7l8T/8szzE8HoLuvRfHxx8XZlVFREREROQyFKJFClHNml6mT0/myy+TqFbNSxqBDOVrvmBElscbpknQmDF4xr9RuBUVEREREZEsKUSLFDLDgD59fEO8x41LxRpg41Ym8y6jLnlOmf8+y+6hz+P1mJc8RkRERERECp5CtEgRCQqCf/7zjyHevTw8wARe5J+XPL7lz6+ysME/2LShECspIiIiIiIZKESLFLELQ7ynTUvmg+ovMI6XLnns0NPvc7j7Azw22srvvxuFWEsREREREQGFaBG/0bu3m9WrE7D8YzQP2d7FS9Yh+Wam0O+zEXRoYeOzz+x4vVkeJiIiIiIiBUAhWsSPBAXBP/6Rxt3rRvBy409xY83yuAHM4vNzMfxrtIeePUPYsCHr40REREREJH8pRIv4oRo1TEYuG0Ds2KmkEZDlMT1YxAKuYf+GeHr0COGRRwI1xFtEREREpIApRIv4scZPXEvS11+R5gjNcn9b1rCULkSaJ/j00wBatAjl448duFyFXFERERERkSuEQrSIn7P27Ixr3ve4wyOy3N+IrSynE9U5wLlzFsaMCaJ58zAmTXKQmlq4dRURERERKekUokWKAU+LFqTM/wFvZGSW++uwlxV0pD47ADh82ML//V8QTZuGMWGCg6SkwqytiIiIiEjJpRAtUkx4o6NJmj8fb7VqWe6vylGW04lmXFxI+sQJC//6VxCNG4fx6qsBnD9fWLUVERERESmZFKJFihFvzZokzpuHp169LPeX5wzL6MxkbmEIXxNKAgBnz1p4/vlAGjUK5z//CdAEZCIiIiIiuaQQLVLMmFWqkPTDD3iaNMlyfyhJ3MIXfM31nKEcc+nDPXxARY4TH2/w8suBNGoUxtNPB3LypMK0iIiIiEhOKESLFENmuXIkzpqFu127yx4XgJM+zOMD7uM4lYmlLf9gPFFJO3jrLQdNmoTx+OOBHD6sMC0iIiIikh0K0SLFValSJH3zDa5rrsn2KW1Zw3ieYDvR7KQ+/04dy7YPf6Fls2AefjiI/fv1liAiIiIicjn6jVmkOAsOJnnKFJzDh+f41Hrs5nFeYQWdOOSuTJfJI3m2+SIevNNkxw69NYiIiIiIZEW/KYsUdw4HKe++S+LPP5M2cuQlZ+++nAqc5k4+4TtzIO9/W5njbW/hs65fsWN5XP7XV0RERESkGLMVdQVEJH94WrbE07IlqePHY/ntN+xz52L/4QesmzfnqJxgUhjI97DpezzXW/g1vA1nh15H1IP98NaqVUC1FxEREREpHhSiRUoaw8DbsCFpDRuSNm4cxpEj2OfNw/bDD9iWL8dwu7NdlBUvTeNj4eNY+Php4qtdhWNoH9x9++Bp1gwsGswiIiIiIlcW/QYsUsKZVavivOcekmfOJH7PHpI/+gjn4MGYYWE5Liv80HYCX3uV0B49CIuOJvDRR7H9/DOkpRVAzUVERERE/I96okWuJBERuIYOxTV0KClOJ7YVK7DNnYtlzjzsJ4/lqCjLiRMEfPIJAZ98ghkWhqtHD9x9+uC69lqIiCiY+ouIiIiIFDH1RItcqRwO3N27k/rqqyTv+I3ExYs5e//jHCt7dY6LMhIScHz3HcH33kt4nTqEDBiA4/33MQ4fLoCKi4iIiIgUHYVoEQHDwNOsGbYX/0XI3pUcWbqJOd3/ywprFzw5fJsw3G5sS5cSNG4c4Y0aEdqpEwEvvYRlyxYwzQK6ARERERGRwqHh3CKSSViTGnT69j5On76dJ8afIHH6enokzaYXPxJCco7Ksv76K9ZffyXwpZfwRkXh6t0bd8+euDt0gNDQAroDEREREZGCoZ5oEbmksDCT60c5eezXPuwaP4VGFU/Rj9l8xF2cpEKOy7McPkzAhx8SMmwY4TVrEhITg+ONN9RLLSIiIiLFhkK0iPyt4GAYNcrJ6s1uer7eg+erfUBljtGelfyXseykXo7LNFwubMuXE/Tss4R17kxY/foE3Xcf9unTMU6fLoC7EBERERHJO4VoEcm2gAC44w4n69cnMGFiGmfqtuEf/JcG7KQB2xnHS6yiHV6MHJdtOXUKx/TpBN93H+F16xLauTMBzz2HdflycDoL4G5ERERERHJOIboEmDVrFgMHDqRGjRpERERw8ODBoq6SlHB2O9x0k4vVqxP55JNkoqM97KQB/2McHVhFZY5xNx8yh76kEpCra1i3bCHw9dcJjYkhvGZNgocNw/HBB1j27tXQbxEREREpMgrRJUBycjLt27fniSeeKOqqyBXGaoVBg1ysWJHIl18m0by5G4CTVGQSdxPDHMpxhsF8w2Ru4TTlcnUdIykJ+48/EjR2LGEtWhDWpAmBjz6KbfZsOH8+P29JREREROSyNDt3CXDjjTcCsG3btiKuiVypLBbo08fNdde5WbzYxiuvBLBqle/tJYlQZjKYmQzGwEtTNtGLH+nFj3RgJXbcOb/eoUMEfPIJAZ98gmm14mndGnf37ri7d8fTtKkv3YuIiIiIFIBi0ROdmprKE088wXXXXUeDBg2IjIykXr169OrViy+++AKXy1XodZo+fTqPPPIIXbt2pUKFCkRERDBlypTLnrNhwwauv/56qlWrRuXKlenZsyczZ84spBqLFDzDgO7d3fzwQxJz5ybSvXvG16aJhY005yX+STeWUIbf6c/3vMMD7KZO7q7p8WCLjSXwP/8htEcPwurUIejOO7F/8QXGsWP5cVsiIiIiIumKRU90UlISH3/8Mc2bN+faa6+lXLlyxMXFsWDBAh588EG+/fZbvv76ayyWwvtM4IUXXuDw4cOULVuWyMhIDh8+fNnjly1bxpAhQwgMDGTw4MGEhoYya9Ys7rjjDo4cOcJDDz1USDUXKRwdOnjo0CGZ9eutvPJKAPPm2TMdk0gYs+nPbPoDUJN96b3UPYxFhJkJOb6u5dw5HN9+i+PbbwHwXHWVr5e6Rw/c7dpBUFDebkxERERErmjFIkSXLl2aQ4cO4XA4Mmx3u90MHDiQRYsWsWDBAnr16nXZcqZNm0b79u2pVq1alvs9Hg8TJ07k3nvvzXStv3r77bepVasW1apV4/XXX+e555675LFut5vRo0djsViYO3cujRs3BmDs2LH06NGD559/ngEDBmSo17PPPssbb7xx2TrExcVddr+IP2jRwsPUqcls22bho48cTJ/uICkp69m791OL9xjFe4zCZrpoy2r6WOZzQ6kfqRW3ASMXE4pZt2/Hun07ARMmYAYG4m7fPj1Uexs08HWfi4iIiIhkU7EYzm2xWLIMtTabjX79+gGwb9++y5Zx9OhRRo8eTb9+/Th06FCm/V6vl1GjRvHkk0/y6aef/m2dunbteskw/lfLli1j//79DB06ND1AA5QqVYoxY8bgdDqZOnVqhnMefPBB1q5de9k/IsVJdLSX115LZfv2eF5+OYUGDTyXPd6NnRV04gnvf6hzbh3lzVOMjZrCttY344msmKs6GKmp2BctIujJJwlr146whg0JevBB7N9+i/H777kqU0RERESuLMWiJ/pSvF4vCxcuBCA6Ovqyx1apUoWPPvqIO+64g5iYGObMmUNUVFR6OSNHjmTGjBnceOON3H333flazxUrVgDQvXv3TPt69OgBwMqVKzNsL1euHOXK5W4mYxF/Fh4O99zj5O67naxaZWXSJAezZtlxuy/fI3yWcrx8eDgvHx5OeJiXMUM3ckel+VTauhDbqlUYaWk5rovl6FEcX3yB44svMA0DT/PmuLt1w92pE55WrSA4OLe3KSIiIiIlVLEK0U6nk1dffRXTNDl37hxLly5l165djBgxgi5duvzt+TExMUyaNIm77rqLfv36MWfOHKpUqcKoUaOYMWMGN9xwA++++26+P1u9d+9eAGrXrp1pX2RkJKGhoX/bk345586d4/Dhw+zfvx+AnTt3cv78eaKioihdunSuyxUpSIZx4bnpFE6eTGXyZAeffurg6NG/f/3FJ1h49usWPEsLOnUax31vn6Nf+DICly7EtmgR1p07c14f08S2fj229evhlVcw7XY8zZrh7tABT/v2uNu08X0CICIiIiJXtGIXov/73/+mf28YBg899BDPPPNMtssYMGAAHo+He+65h5iYGJo1a8bMmTMZMmQIEydOLJDJyeLj4wEIv8Qv4GFhYenH5MYPP/zAAw88kP79DTfcAMCECRMYMWJErssVKSyRkSaPP57Go4+m8eOPNiZNcrBoUeaJyLKyfLmN5cvLU7HiIG67rS+3zXRSxXsY26JFvj9LlmDJxfwBhsuFbe1abGvXwuuvY1oseBo39gXq9u3xtG+PWaZMjssVERERkeKtWIXo0NBQ4uLi8Hq9HD9+nPnz5/Pvf/+bX375hRkzZlwypP7V4MGDcbvd3HvvvRw4cIC+ffvywQcfYC2ma8uOGDFCYVlKBJsN+vZ107evm717LXz8sYMpU+zExf39h1snTlj4738DeeWVAPr0qcfdd1en8ye3Yng9WDdswLZwIbbFi7H+8guG15vjuhleL7ZNm7Bt2kTAu+8C4ImOTg/U7vbtMSvm7lltERERESk+isXEYn9lsVioUqUKd911F2+++SarV6/m1Vdfzfb5pmmybNmy9O937NjBqVOnCqKqwMUe6Ev1NickJGT7AwCRK0Xt2l7+859Utm1L4J13kmnWzJ2t8zweg9mz7QwYEErr1qFM/CCIs3VbkfaPf5D044/E79tH0mef4bztNrxVq+apjtZt2wj46COC77yT8AYNCG3RgqCHHsI+bRpGFhMYioiIiEjxVyxD9J9169YNuDh5198xTZPRo0fzxRdfMHjwYD744AMOHDhATEwMx48fL5A6XngW+sKz0X928uRJEhMTqVWrVoFcW6S4Cw6Gm292sXhxEosWJTJihJPAwOwtdbV7t5V//jOIq64K56GHgti0yQIREbgHDCDlzTdJ+PVXEn75hZSXXsJ17bWYeZxIzLp3L47PPyd45EjCGzf2zf59773YJ0/GsmcP5GKJLhERERHxL8U+RJ84cQIAu/3vn580TZNHHnmEyZMnM2jQID788ENuuOEG3n//ffbv309MTEx6efmpQ4cOACxatCjTvguzi184RkQurXlzDxMmpLB9ewIvvJBCrVqXXybrgpQUg88/d9C1axg9e4Ywdaqd1FTAMPDWrYtz5EiSZ8wg/sABEhcsIOW553D16oWZxxEiliNHcMyYQfDDDxPWsiVh9esTdPvtOD78EMtvv0EuhpWLiIiISNEqFiF6x44dJCcnZ9qenJzMv/71LwCuueaay5ZhmiZjxozhs88+Sw/QF56BHjp0aIYgffLkyXytf5cuXahRowZff/01W7ZsSd9+/vx5XnvtNRwOBzfeeGO+XlOkJCtd2uTBB52sW5fIzJlJ9O3rwmLJXi/vunU2Ro0KJjo6jKefDmT//j+9DToceFq1wjl6NMnTpxO/fz8JS5eSMn48rn798OZxIjHLqVM4vvuOoMcfJ6xDB8Jq1yZ4+HAc77yDdeNGcGdvyLqIiIiIFJ1iMbHYzJkzeffdd2nbti3VqlUjLCyMY8eO8fPPP/P777/Trl077r///suWcfz4cebMmcPAgQP58MMPsdky3vrQoUMxTZORI0eyZMkShg0bdtnyJk+eTGxsLADbtm0D4PPPP08fVt6uXTtuvfVWAGw2G2+99RZDhgyhb9++DB48mNDQUGbNmsXhw4d5/vnnqV69eq5+NiJXMosFunVz062bmyNHDD791MHkyQ5Onfr7zwd//93CW28F8PbbDnr0cHPXXU6uvdZNhvkFrVa8TZrgbNIE56hR4PVi2bUL26pVWFeuxLZyJZY8jF6xnDuH5YcfsP/wAwBmWBjuNm0uzgDevDk4HLkuX0RERETynxEXF+f3D+lt3LiRTz/9lLVr13Ls2DGSkpIIDw/n6quvZsiQIdx8882ZQnFWDhw4QNWqVS977J49e6hTp87fljVq1CimTp16yf033XQTEydOzLBt/fr1jB8/nrVr1+JyuYiOjuaBBx5g8ODBf3u9K0FqaiqHDx8mKiqKwMDAoq6OUDzbxOmEOXPsTJrkYOXKnH1OGBXl5Y47nAwb5qRKlWy8NZomlgMH0gO1bdUqLAcP5rLmWRQfGIinVSvc7dvj7tCBpIYNOXz2bLFqj5KuOL5GSjq1if9Rm/gXtYf/UZsUP8UiRMuVQW8g/qe4t8n27b5lsqZNc5CQYOTo3Nat3QwY4GLAABdVq2b/bdI4csTXU71qle/vXbtyWu1LMu12kho0wBodjVGtGt6qVTErV8ZbpQreKlUgPByMnN2n5E1xf42URGoT/6M28S9qD/+jNil+FKLFb+gNxP+UlDZJSICvvnLw0UcOtm3L+XrwLVteDNTVquXsLdM4dQprbOzFnurffsMooFm6zdDQ9ED953Bt/vG3t3JlX9CWfFNSXiMlidrE/6hN/Ivaw/+oTYqfYvFMtIhIXoSFwZ13OrnjDierV1v5+GMH331nx+XKXq/tunU21q2z8dRTQbRo4WbgQBf9+7uoXv3vw7BZoQLuAQNwDxjg2xAXhy02Nr232rppE4Yne7OM/x0jMRHrzp1Yd+68dH3Cw/H+EbDNP4K1t0oVzKpV078mNDRf6iMiIiJSEhV4iI6Li+PYsWPUrl2bgICAgr6ciMglGQa0a+ehXbsUXnwxlc8/d/Dxxw6OHMn+QgXr19tYv94XqJs18wXqAQNc1KiRzd7liAjc112H+7rrfN8nJmJbu9Y3/HvlSqzr12M4nbm4u+wx4uOxxsdj3bHjkseYpUpl2aP95+BNSEiB1VFERETEn+U5RG/evJk5c+bQrl07unfvnr49JSWFBx98kJkzZwJQqlQp3njjDQZc6I0RESlC5cubjBmTxujRaSxYYGPSJAc//2zDNLP/TPHGjTY2brTxzDNBNG16IVC7qVkzB+s/h4bi7t4dd/fupAGkpGBdvx7bhVC9di1GSkqO7y8vjPPnsZ4/j/WPlQey4o2IuDhM/C+92t569TArVSrEGouIiIgUnjyH6C+++IJJkybx1VdfZdj+4osv8u2336Z/HxcXxz333EPdunWJjo7O62VFRPKF1Qq9e7vp3dvNiRMGs2fb+e47O6tWWXMUqDdtsrFpk41nn4XGjT0MHOhi4EAXtWrlIFADBAXh6dgRT8eOvlDtdGLdvBnrqlVYli/H3LGDgNOnMdLSclZuPrPExUFcHNbffstyv7dyZTwtWuBu1QpPixZ4mjZV77WIiIiUCHmeWKxDhw7s37+fI0eOYLH4hkQ6nU7q1KlDamoqU6ZMoXXr1owfP57333+fW265hbfeeitfKi8liyZV8D9XcpucPJkxUHu9uZv1umHDi4G6Tp0cBuq/SG+PqlUJSkrCOHIEy9GjWI4dwzh61Pf1H3+MY8cwXK48XS8/mVYr3quuwt2ypS9Ut2yJt35932LfxdiV/BrxV2oT/6M28S9qD/+jNil+8twTferUKSpVqpQeoAHWrl1LQkIC/fv355prrgHgmWee4YsvvmDlypV5vaSISIGLjDS5+24nd9/t5NQpgzlzfIF6xYqcBeqtW61s3WrlhRcCufrqi4G6bt08BGrDwCxXDrNcObxNm2Z9jNeLceaML1D/JVxnCNpud+7rkZMqezxYt27FunUrfPopAGZYGJ5mzTIEazMyslDqIyIiIpJbeQ7RcXFxVK9ePcO2tWvXYhgGPXr0SN8WFBREjRo12Lt3b14vKSJSqCpUMLnzTid33unk9GlfoP7+exvLl9vweLIfqH/7zcpvv1n5z38CiY72MGCAL1DXr5+3HuosWSyYFSrgqVABmjXL+hivF+PUKV9P9l97tY8dw3LkCMbx4/k2e/hfGQkJ2JYtw7Zs2cUqVa2aIVR7mjSB4OACub6IiIhIbuQ5RAcFBXHmzJkM22JjYwFo06ZNhu0OhyNDj7WISHFTvrzJHXf4lss6c8Zg7lwb331nZ9mynAXqbdusbNtmZfz4QK666mKgbtCgAAL1pVgsmBUr4qlYEZo3z/oYj8cXtC/Xo338OIY3f+ptOXIEx5Ej8N13wB/DwK++OuMw8Lp1i/0wcBERESm+8hyi69Wrx4YNG9i+fTtXXXUVZ8+eZfny5ZQtW5b69etnOPb48eOUK1cur5cUEfEL5cqZ3Habi9tuc3H2rC9Qf/+9naVLbbjd2Q/U27db2b7dyksvBVK//sVAfdVVXozcPYqdf6xWzEqV8FSqBC1bZn2M241x8qQvVO/Zg3XDBqzr1mHdujXPw8UNjwfrli1Yt2yBjz8GfGtdu1u0uNhb3bIlpv5vERERkUKS5xA9cOBA1q9fz/XXX8+AAQNYvHgxTqeTwYMHZzju8OHDnDhxgq5du+b1kiIifqdsWZNbb3Vx660ufv/9YqBesiRngXrnTiv/+5+V//0vkHr1fIF6wAAXV19diD3UOWWzYVapgqdKFTytW+MaPty3PSXFF4DXrfMt27VuHZZDh/J8OSM+HvvixdgXL07f5q1ePeMw8MaNQZOziIiISAHIc4i+9957mTdvHqtWreLdd98FoG7duowbNy7DcRfWi+7UqVNeLyki4tfKlDG55RYXt9zi4tw5gx9+8AXqxYttuFzZD9S7dll5+WUrL78cSJ06Hvr1s9KqVRBVqxZg5fNTUBCeNm3w/OnRHuPUKazr1/tC9S+/YN24ESM+Ps+Xshw8iOPgQfjmGwBMux1Pw4a+QH1hGHjt2hR9176IiIgUd3kO0Q6Hg9mzZzNv3jx2795NVFQUffv2zTQ9u9VqZeTIkQwYMCCvlxQRKTZKlzYZMcLFiBEu4uLghx/sfP+9nUWLchao9+yx8sYbocDV1KrlZvBgNwMH+nqoi1MuNCtUwH3ddbivu863DrbXi2XXroy91du25XkyM8PlwrZxI7aNG+HDDwHwRkT4AnWrVrh698bbpIlCtYiIiORYnteJFskvWiPP/6hNCk5cHMyf71s2a9EiG05n7sJc3bq+ZbMGDfKTZ6jzQ3Iy1k2b0kO1df16LEeO5PtlvFFRuPr1wxUT4+stt1pzXIZeI/5HbeJ/1Cb+Re3hf9QmxY9CtPgNvYH4H7VJ4Th/3heov//ezsKFNtLScpeE69e/GKgLdZbvQmCcOJGht9q6cSNGYmK+le8tXx5X3764Y2Jwd+oEDke2ztNrxP+oTfyP2sS/qD38j9qk+MlziD5z5gy//fYbVapUoU6dOhn2ffLJJ3z00UccP36cFi1a8OKLL1K3bt08VVhKLr2B+B+1SeGLj4cff/T1UP/8c+4D9VVX+QJ1ga1DXdQ8Hiw7d2Jdtw7b+vVY163Dsn17viy1ZYaH4+rdG1dMDO4ePS67TrVeI/5HbeJ/1Cb+Re3hf9QmxU+eQ/QLL7zAa6+9xoQJE7jpppvSt3/66aeMGTMG07xYfGRkJKtWraJMmTJ5uaSUUHoD8T9qk6KVkOAL1N9/b2fBAhupqbkL1NHRHgYN8gXqunVLYKC+IDEx8zDwY8fyVKQZFIS7Z09cMTG4rr0WIiIy7NdrxP+oTfyP2sS/qD38j9qk+MnzxGLLly/HarUSExOTYftrr70GwEMPPUSbNm2YMGECq1ev5t133+XJJ5/M62VFREq8sDAYOtTF0KEuEhNhzhyT6dNNYmMjchSot22zsm2blf/8J5Crr/YF6kGDXNSuXcICdWgono4d8XTsiPOPTcaxYxl6q60bNmCkpGS7SCMlBfvs2dhnz8a023F36eLroe7TB7N8+YK5DxEREfFree6Jjo6OxmKxsHXr1vRtv/76K507d6Zt27bMmzcPgJMnT9KwYUMaNGjA8uXL81ZrKZH0KZz/UZv4lwvtUbp0NZYuDWXmTF8PdW6HfDdqdLGHulatEhaoLyU5GdvChb5gPH9+rpfXMi0WPG3bknzddRxo2pTIVq30GvETet/yP2oT/6L28D9qk+Inzz3Rv//+Ow0bNsywbfXq1QD06dMnfVtkZCS1atXiwIEDeb2kiMgVLTTUZMgQF0OGuNKfoZ450/cMdU5m+f71Vyu//mrl3/8OpEkTD4MGORk40EWNGiV4vsngYN/kYTExpDid2JYvxzZ7Nva5c7GcPp3tYgyvF9uqVYSvWkVjwNWkCZ4BA3D164e3Xr2Cq7+IiIgUOUueC7BYSPzLDKlr167FMAzatm2bYXt4eDhOpxMREckf4eFw/fUuvvwymd2743n//WR693bhcOQsCG/ebOXZZ4No2jScbt1CeOstBwcPloT1si7D4cDdowepb7xBwo4dJM6bR9r99+ONispxUfbNmwn8978Ja92a0DZtCHjhBSybNoFZgj+QEBERuULlOURXq1aNffv2ce7cOQBcLheLFi0iKCiIZs2aZTj27NmzlC1bNq+XFBGRLJQqBcOGuZg2LZldu+KZODGZXr1c2O05C3IbN9p4+ukgmjQJp3v3EN5+28GhQyU8UFuteNq1I/XFF0nYsoWEJUtIfewxPPXr57yonTsJfOUVwrp2JaxxYwKfeAJrbCx4PAVQcRERESlseQ7R3bt3x+VycddddzFv3jweeughfv/9d3r06IHNdnG0+Pnz5zlw4ABVqlTJ6yVFRORvRETATTe5mD7d10M9YUIy117rwmbLWaDesMHGU08F0bhxOD17hvDOOw4OHy7hgdow8DZtStqTT5K4Zg0Ja9aQ+tRTuJs2zXFRlsOHCXj3XUKvu46wq64i8NFHsS1aBBqVJSIiUmzl+ZnoRx55hG+//ZbFixezZMkSTNMkMDCQcePGZThu/vz5mKZJu3bt8npJERHJgYgIGDHCxYgRLs6dM5g718Z339lZssSG2539QLxunY1162w8+WQQrVu7GTjQxYABLqpUKdlDlr3165NWvz5p//d/GIcOYZ8zB/vs2VhXr8bIwXBty6lTBHzyCQGffIJZqtTFtai7d7/sWtQiIiLiX/LcE12hQgUWLVrEfffdR/fu3bn99ttZsmQJV199dYbjYmNjadiwIb169crrJUVEJJdKlza5+WYXX3+dzK5dCbz1VjLdu7uwWnMWhNeutfHEE0FcfXU4vXuH8N57Do4dK+E91IBZrRrO++8nad48EnbuJP7llznfti2mLWefSRvnz+OYPp2Qm28mvE4dgm+9FftXX8H58wVUcxEREckveV7iSiS/aHp//6M28S8F2R5nzxrMmeProV62zIbHk/NAbBgmbdt66N/fRc+eburU8WKU8Fx9oU2qhYcTumQJ9tmzsS1cmKO1qP/MtNtxd+2K+9prcXfvjrdWLUr8DzGf6X3L/6hN/Ivaw/+oTYqfPA/nFhGR4q9sWZPbbnNx220uzpzxBeqZMx0sX27F681eiDNNg9hYG7GxNv75T4iK8tK9u5vu3V106eImIqJg76EomaVK4Ro2DNewYXlai9pwubAvWIB9wQIAvFFRuLt18/3p0gWzTJmCugURERHJpnwN0adOnWLx4sXs3r2bhIQEwsLCqFevHt26daN8+fL5eSkRESkg5cqZ3H67i9tvd3H6tMHs2b51qFeuzH6gBjh82MJnnzn47DMHFotJy5aeP0K1m+bNPeRwBHTxkU9rUYNvYjLH5Mk4Jk/GNAw8TZv6AnXXrnjatIGAgAK6CREREbmUfBnOnZaWxlNPPcVnn32Gy+XKtN9ut3PHHXfw3HPPEaD/8OUSNJTF/6hN/EtRt8fJkxcD9apVVkwz98OMS5Uy6drV10vdvbubqKji+WRRjtrE48G6Zo2vh3r2bCxHjuTp2mZwMO727dN7qr1XXaWh3xT960QyU5v4F7WH/1GbFD95DtFer5ehQ4emz8xdvnx56tatS8WKFTlx4gS7d+/m9OnTGIZB9+7d+eqrrzD0n7xkQW8g/kdt4l/8qT1OnDCYNcvOd9/ZiY3NW6AGqFfvYi91hw5uQkLyqaIFLNdtYppYNm/2BepZs7Du3p3nungrVsTdpUt6qDYjI/NcZnHkT68T8VGb+Be1h/9RmxQ/eQ7RkydPZvTo0YSHh/PCCy9w0003ZVgf2uPxMHXqVJ566inOnz/PW2+9xc0335znikvJozcQ/6M28S/+2h7Hj18M1GvW5GzId1YcDpN27TzpvdQNG/rvBGX51SaWnTt9k5LNmYN18+YcLZ11KZ7o6IvPU7dvX3KX0YqLw7pvH5Y9e7Ds3Yu5fz/xFgu2u+7C1qpVUddO8N/3riuV2sP/qE2KnzyH6L59+xIbG8vMmTPp0qXLJY9bunQpAwcOpH379sydOzcvl5QSSm8g/kdt4l+KQ3ucO2ewZImNRYt8f44ezfNKilSo4KVbNzc9erjp1s1N+fL+M/S7INrEOHsW29Kl2BYvxrZ4cZ6HfQOYDgeetm1xd+uGq1s3vI0bgyXvbVNokpOx7NuHZe9erHv3+gLzheB85swlT3P17k3a2LF4mjcvxMrKXxWH964ridrD/6hNip88h+gaNWpQunRpNm7c+LfHNmvWjHPnznHgwIG8XFJKKL2B+B+1iX8pbu1hmrBzp4WFC20sXmxj5UobKSl571Ju3NhDjx6+Xuo2bTw4HPlQ2Vwq8DYxTSx79qQHatuKFRgJCXku1lumTMah31FR+VDZPHI6sRw8mN6jnB6Y9+7FcvRonop2XXONL0yrZ7pIFLf3rpJO7eF/1CbFT57nRk1JSaFWrVrZOrZ06dIcP348r5cUEZFiwDCgQQMvDRo4eeABJ6mpEBvr66FeuNDGtm3WXJW7ZYuVLVusvP46hISYdOzo66Xu3t1N7dr+O/Q7VwwDb926OOvWxXnvveByYV23zheolyzBun49hseT42Itv/+OY+ZMHDNnAuCpU+fi0O+OHSE8PL/vxMfjwThy5GI4/nOP8qFDubqX7LiwbJire3dfmG7btkCuIyIiV4Y890Q3btyYc+fOsWvXLoKCgi55XHJyMvXr16d06dJs2bIlL5eUEkqfwvkftYl/KWntcfy4weLFvlC9eLGNs2fzPry4WjUvPXq46NbNTZcubkqVyoeKXkaRt8n5874ltP7oqbbu25fnIk2rFU+rVri7dsXdrRueFi3I0Xpkpolx8mR6QLbu2XMxLO/bh+F05rmOeeXu3JnUcePwdOhQ1FW5IhT560QyUHv4H7VJ8ZPnnuhOnToxdepUnnjiCV5//fVLHvfEE0+QlJTEgAED8npJEREpASpVMhk+3MXw4S68XtiyxcLChXYWLrSxdq0VtzvnXcqHDln45JMAPvkkAKvVpFUrT/rz1M2aebDmrvPbf5UqhbtfP9z9+gFgHDyIbcmS9J5qS1xcjos0PB5sq1djW70aXnoJMzwcd6dOF5fSqlULDAPj3LmLQ6//+Nu6d68vKCcm5vON5i/bsmWELluGu0MHUseOxdO5s5YHExGRbMtzT/SuXbvo1KkTLpeL6OhoRo4cSXR0NBUqVODUqVNs27aNiRMnsn37dhwOB8uWLaNevXr5VX8pQfQpnP9Rm/iXK6k94uNhxYqLQ7/37897+o2I8P6xNrXvT9WqeZ+gzK/bxOPBunnzxV7qNWswXK48F+utUsU30de5c/lQydwxbTa8NWrgrV3b96dOHTx/fJ1qGLhffZWKX32FJZvPj7vbtSNt7FjcXbsqTBcAv36dXIHUHv5HbVL85DlEA8ycOZP777+f1NTULNeANk2TwMBAJk6cyMCBA/N6OSmh9Abif9Qm/uVKbo/9+y3pgXr5chsJCXkPOvXr+3qp87I2dbFqk8REbKtWXXyeevv2oq7RZZmGgVm1qi8c16mDt1Yt39916uCNigK7PcvzLrRJtbAwwj/5hICJEzHi47N1TXerVr4w3bOnwnQ+KlavkyuA2sP/qE2Kn3wJ0QC7d+/mzTff5Oeff+bkyZPp2yMjI7n22mt56KGHqFu3bn5cSkoovYH4H7WJf1F7+Lhc8Msv1vRQvWmTFdPM+9rUbdv61qbu1s1No0bebK0AVZzbxDh+POPQ71OniqQe3sjI9IDs+VPPsrdmTcjFzzRTm5w/T8AHH+CYMCHbw9vdzZv7wnSvXgrT+aA4v05KIrWH/1GbFD/5FqL/LD4+nsTEREJDQwn/0wyfXbp04fz582zatCm/LyklgN5A/I/axL+oPbJ29qxvbeoLS2kdP573CcrKl/etTX3hT8WKWf9XWWLaxDSx/PZbeqC2rVqFkZKSf8WXKoWnTp3Mw69r1cr3mcAv2Sbx8QR89BGOd97B8vvv2SrL06QJqWPH4u7TR2E6D0rM66SEUHv4H7VJ8VMgIfpSateuzblz5/g9m/95yZVFbyD+R23iX9Qef880Yft239rUixbZWLXKRlpa3sNPdLSH7t19E5S1bevmwmIUJbZNUlOxrlmDbfFi7IsXY928+W9PMYOD8daqlTEs/xGYzTJlCi2E/m2bJCbimDSJgLffxnLmTLbK9DRsSOrjj+OOiSFbQxQkgxL7Oimm1B7+R21S/ChEi9/QG4j/UZv4F7VHzqWkwKpVF3upt2/P+wRlgYEm7dv7eqg7dkwmJGQ/1aqV7DYxzp7FtnQptsWLsezfjxka6utN/tOzymalSn7RW5vt10lSEo5PPiHgrbeyPZTdEx1N2uOP4+rfn5I31XvB0XuXf1F7+B+1SfGT5yWuRERE/FVQEPTo4etBBjh61Lc29YU/v/+e817F1FSDRYvsLFpkB4IoVy6M7t09XHONSdeubsqXL7TPpguNWbYsrsGDcQ0eXNRVyT8hITgffBDnXXfh+PRTAt58E8uJE5c9xbptG8F33IGnfn1fmB40SGFaROQKpDFJIiJyxahSxeTmm11MmpTCnj0JLFmSwFNPpdKhgxubLXfh98wZBzNmBHHPPcHUrRtO586hPPdcAMuWWUlLy+cbkPwXFIRz1CgSNm0i5eWXfUt4/Q3rzp0E3303oW3aYJ82DdzuQqioiIj4C4VoERG5Ilks0LSpl//7vzTmzk1i//54pk1L4t5706hTx5PrcrdssfL664H07x9KzZrh3HBDMBMnOti1y4JZ8jqpS47AQJz33EPChg2kvPYa3qpV//YU6549BI8cSWjr1tinTPFNHV/cmSb6hyoicnkazi0iIgKEhUHv3m569/b1Kh486Jv1e9EiO0uW2Dh/PufP+yYnG/z0k52ffvKtaVy1qjd9bequXd2ULq2w4ncCAnDeeSfOm2/GPnUqga++iuXQocueYt23j+AHHsD7v/+R+n//h+vGG8HhKKQK50JqKpZDh7AcPIjlwIGMfx88CCkpeGvXxjVkCM4bb8SMiirqGouI+BVNLCZ+Q5Mq+B+1iX9RexQdjwc2brSmT1D2yy9WPJ68TaJlGCbNm3vSQ3WrVh7s9nyq8BUs318nLhf26dMJePVVrPv3Z+sUb1QUaWPG4Bw+HAIC8l6HnPJ6MY4dSw/Ffw7IloMHsRw/nu2iTMPA07kzzuHDccXEQHBwjquj9y7/ovbwP2qT4ifHIfq///1vri/25ptvkpqaqhAtWdIbiP9Rm/gXtYf/OH8eli+3sWCBwc8/Wzh6NO/tERZm0rGjbxK0a691Ua2aeqlzo8BeJ2439q++IuCVV7Du3ZutU7xVqpD2yCM4b7kF8vs1GxeXZS+y5cABLIcPYzid+Xs9wAwLwzVoEM7hw/G0aZPt2dj13uVf1B7+R21S/OQ4RJcuXRojl0tYmKaJYRgK0ZIlvYH4H7WJf1F7+J8LbeLx1GDlyhAWLbKxbJmNhIS8L/XUpImHmBgXMTEu6tf35kNtrwwF/jrxeLB/+y0BL7+MddeubJ3irVSJtNGjcd52G+mLjP+d1FQshw9nDsp//G3Ex+fhJvLOU6sWruHDcQ4b9rfDvfXe5V/UHv5HbVL85DhE9+3bN9ch+oI5c+bk6XwpmfQG4n/UJv5F7eF/smoTlwvWr7849HvDBiteb97+36xX72KgbtLE6w/LMfutQnudeDzYv//eF6a3b8/WKd4KFUh7+GGcd94JgYEYx49n7kU+dMj3dw6GXBcl0zBwd+mCa8QIXH37ZjncW+9d/kXt4X/UJsVPoT4TLXI5egPxP2oT/6L28D/ZaZNz5wyWLbOyaJGdhQttHDmSt4Uxqlb10q+fL1C3bevRMsV/UeivE68X2+zZBP7vf1h/+y1bp5ihoeB0FsiQ69zyli6Nt3p1vDVqYCQnY1u4EMOTs1nqzfDwi8O9W7dOH+6t9y7/ovbwP2qT4kezc4uIiBSg0qVNBgxwM2CAG9OEPXssLFpkY9EiGytW2EhKylm38pEjFt57L4D33gugXDkvffu6iYlx0bmz268nhC6xLBbcAwaQGBOD7YcffGF6y5bLnmIkJhZS5S4yAwJ8IfmPoOytVu3i19WrQ6lSGet48iT2GTNwfPlltnvajfh4HJ99huOzz/DUru0b7n3jjVC2bEHckohIkVFPtPgNfQrnf9Qm/kXt4X/y2iZOJ6xda2XxYhsLF9rYvNmKaeZurHZ4uEnv3i769XPRo4ebkJBcFVPsFfnrxDSxzZ9PwP/+h23jxsK7rGFgVq6cMSj/6WszMtK3OHqOCzaxbN6MY8oU7F99hSUuLsf1cnbpwtGePQkZMYLA0qVzXgfJV0X+GpFM1CbFj0K0+A29gfgftYl/UXv4n/xuk9OnDebNszF7tm9tapcrd4E6KMike3dfD3Xv3i4iIvJctWLDb14npoltwQJfmF63Ll+K9EZEYGYRkL3Vq+ONiir45bTS0rDNn4/jyy+xLViA4c3ZhHfesDBcQ4bgGj4cT6tW2Z7dW/KX37xGJJ3apPhRiBa/oTcQ/6M28S9qD/9TkG1y/jz89JOd2bPt/PyzjeTk3AUOm82kc2c3MTFu+vRxERlZsv/b97vXiWliW7yYgP/+F9uaNZc/1OHIHI6rVbs45NqPPg0xTpy4ONx7x44cn++pW/fi7N6VKxdADeVS/O41ImqTYkghWvyG3kD8j9rEv6g9/E9htUlyMixaZGPOHDvz5tk5fz53gdowTNq08dCvn2/Yd40aJe9XAL99nZgm1mXLsM+Zg2X/fsxy5TKG5Ro1MCtWzN2Q66Jkmlg3bsT+5Zc4vvoK4/z5nJ1useDu1g3X8OG4+vTJ/hJgkmt++xq5gqlNih+FaPEbegPxP2oT/6L28D9F0SYuF6xYYWP2bF+oPnUq96GrceOMa1GXhNG1ep0UodRU33DvKVN8s3vncLi3GR6Oc+hQ33DvFi003LuA6DXif9QmxU8x+7hTRETkyma3Q7dubl57LZXt2xOYPz+RBx5Io1q1nAUWgC1brPznP4G0bRtG69ahPPdcABs2WDH18brkRmAg7oEDSf7qKxJ++42U557DU79+tk834uMJ+PhjQnv2JLRNGxxvvIFRTNbLFpEri0K0iIhIMWW1Qtu2Hv7zn1Q2b05g6dIEHnsslQYNcra+L8Du3VZefz2Q7t1DadQojHHjAlmxwkoOlwoWAcCsVAnn6NEkrl7N2R9+4NSQIXj/sozW5Vh37SLo2WcJu/pqgocOxf7tt5CaWoA1FhHJPoVoERGREsAwoEkTL08+mcbq1YmsXZvA00+n0ry5O8dlHTli4f33A+jXL5T69cN4+OEgFiywkZZWABWXks0wcDdvzqF//IPTmzeT/PHHuHr2xMzms9+G14v9558JvvNOwuvXJ/D//g/r+vVouISIFCVbUVdARERE8l+9el7GjEljzJg0Dh82mDvXN9N3bKwVrzf7z5qeOWNh8mQHkyc7CA83ufZa3zPUPXteuWtRSy4FBuIaPBjX4MEYx45dnN17165snW6cP0/ApEkETJqEt1o1PLVqYVapgrdKFbxVq/rWyf7je8LDC/hmRORKphAtIiJSwkVFmYwc6WTkSGee1qKOjzf4+msHX3/tICjIpGdPNwMGuLj2Wpcyi+SIWbkyzkcewTl6NNb1632ze3/9NUZ8fLbOtxw6hOXQoUuXHx6eHqjTg/aFsF2lCt7KlSE4OL9uR0SuMArRIiIiV5Dy5U1uvdXFrbe68rQWdUqKwezZvnMdDpPu3d307++iTx+XPy1nLP7OMPC0bImnZUtS//Mf7D/8gP3LL7EtWoSRhyHbRnw81vh4rNu3X/IYb+nSGXuy/xq2K1WCgIBc10FESi6FaBERkStUqVJw/fUurr/elae1qJ1Og/nz7cyfb8dmM+nSxddD3bevm7Jl9eyqZFNQEK4hQ3ANGYJx9CiOGTOwT5mCdc+eArmc5dw5OHcO69atlzzGW6EC3sqVLxm2zUqVwFbEv057veB0Qloahsvl+9rpTP/a+OP7C9sciYmUOnsWa/v2kIPZ00XkIoXoEmDWrFl8/PHHbNq0ibi4ODZv3kz16tWLuloiIlKMBAdDv35u+vVz43Kl5HotarfbYOFCOwsX2nn0UZMOHTwMGOCiXz8XkZEK1JI9ZpUqpD36KGmPPIL1l198w72//Tbbw73zi+XUKSynTsGmTVnX02LBrFjRF6j/9Ey2WbYsuFy+IJuW5vv6Qpj909dZbcPlwvjjnPRj/hKIM2zL4RT6IUDpP772VquGu2tX35/OnTHLlcvTz0vkSmHExcXpf7Ribtq0aRw6dIiIiAjGjh1bbEO0Fpr3P2oT/6L28D9XQpt4PPDLL9b0oduHDuVuYQ/DMGnb1kP//r6JyapWLZhfP66ENilu8q1NUlKwz52LbdEiLAcPYhw9iuXYMV+glHzhadjQF6i7dMHdrh2EhhZ1la4Iet8qfhSiS5Bt27bRvn17hWjJN2oT/6L28D9XWpuYJmzebGHWLDvff29n715rrstq2dI35DsmxkWNGvn3q8iV1ibFQYG2ideLcfasL1AfOYLl6FEsR4/6vv9jm3H8eI57awVMux1Pq1a+QN21K57mzcFuL+pqlUh63yp+isVw7mPHjvHdd9+xYMECdu/ezcmTJyldujRt2rRh9OjRtGzZstDrNH36dGJjY9m0aRPbtm3D6XQyYcIERowYcclzNmzYwPjx41mzZg1ut5vo6GgeeOABBg0aVIg1FxERyR3DgKZNvTRtmsZTT6WxbZsvUM+aZWf79pwF6nXrbKxbZ+Opp4Jo0sTXQz1ggIs6dbwFVHspkSwWzPLlMcuXx9u0adbHeDwYJ09eDNcXwvaxY+lh2zhxIk8TmZVEhsuFbdUqbKtWwfjxmGFhuNu3Tx/+7W3QwPemIHIFKhYh+oMPPuCNN96gZs2adOvWjXLlyrF3717mzp3L3Llz+eijjxg8eHCh1umFF17g8OHDlC1blsjISA4fPnzZ45ctW8aQIUMIDAxk8ODBhIaGMmvWLO644w6OHDnCQw89VEg1FxERyTvDgKuv9nL11Wn8859p7Np1MVBv2ZKzQL15s5XNm608/3wg0dG+QN2/v4urrvLqd3TJO6sVs3JlPJUrQ6tWWR/jcmEcP56xJ/uvPdtnzhRuvf2MkZCA/ccfsf/4IwDeyEhfL3Xnzri7dsWsWrWIayhSeIrFcO5Zs2ZRpkwZOnbsmGH7qlWrGDBgACEhIezcuZOAv1mGYNq0abRv355q1aplud/j8TBx4kTuvfdeHA7HZctasmQJtWrVolq1arz++us899xzl+yJdrvdtGrVimPHjrFgwQIaN24MwPnz5+nRoweHDh1i3bp1Ger17LPP8sYbb1y2DnFxcRm+13BuyW9qE/+i9vA/apOsHThgpA/5Xr8+95/X1617MVA3bpy9QK028T8lpk1SU7EcP45xqWHjR49i+cvvZpdj2mzgcIDdjulwgMOBabdf3BYQkHH/H/vSj/3rtgtfBwRc/PpCeRe+DgjACZzbvp1K27cTsHw5liNH8uXH46lTxxeq/wjWWusu+0rMa+QKUix6ovv375/l9vbt29OpUycWLVrEtm3baNas2SXLOHr0KKNHjyYyMpI5c+ZkCtJer5dRo0YxY8YMHA4H995772Xr1LVr12zXf9myZezfv58RI0akB2iAUqVKMWbMGO6//36mTp3KuHHj0vc9+OCDDB8+PNvXEBER8Rc1apg8/LCThx92cvjwxfWkV6+2YprZ71revdvKq69aefXVQGrU8NC/v28t6hYtPOqhlsIXGIi3Zk2oWZNLPmGdlITl6FFITvYF2YCAzEH2QuC15n5Ogbxwpqbye40ahNx7L4EBAVj27cO2dCm2JUuwLluWow8C/sy6Zw/WPXsImDQJ02LB06RJ+tBvT5s2oHAoJUixCNGXY/9jggPr37wRValShY8++og77riDmJgY5syZQ1RUFOAL0CNHjmTGjBnceOON3H333flaxxUrVgDQvXv3TPt69OgBwMqVKzNsL1euHOW0zICIiBRzUVEm99/v5P77nZw4YTBnjq+HeuVKK15v9pPwgQNW3nrLyltvBVC1qpd+/XzPULdp48GSuwnDRfJfSAjeevWKuhbZZxh4a9fGWbs2zjvvBI8Hy6+/YluyxPdn9WqM1NScF+v1Ytu4EdvGjfD665gBAXjatk2f+dvTpEmRfYggkh+KdYg+fPgwS5YsoWLFilx99dV/e3xMTAyTJk3irrvuol+/fsyZM4cqVaqk90DfcMMNvPvuu1jy+X/jvXv3AlC7du1M+yIjIwkNDWXfvn25Lv/cuXMcPnyY/fv3A7Bz507Onz9PVFQUpUuX/puzRURECkfFiiZ33+3k7rudnDljMHeujVmz7CxdasPtzn6gPnLEwnvvBfDeewFERnqJifHN8t2hg2ZgFskTqxVv06Y4mzbF+cgjkJqKdc0aX0/10qVYN27E8OZ88j8jLS29DACzVCnfs9R/zPztrV1bk5RJsVJsQ7TL5eK+++4jLS2NZ5999m97oi8YMGAAHo+He+65h5iYGJo1a8bMmTMZMmQIEydOzPcADRAfHw9AeHh4lvvDwsLSj8mNH374gQceeCD9+xtuuAHgb2cLFxERKSrlypncdpuL225zERcHP/zgm5Rs0SIbTmf2f5k+edLCRx8F8NFHAZQt66V3bzutW4fTv79Gj4rkWWAgni5d8HTpQhpAXBy25cuxLVvmG/69e3euijXOn8c+ezb22bMB8Fatmj5BmbtLF8zIyPy7B5ECUCxDtNfr5f7772fVqlXcdttt3HjjjTk6f/Dgwbjdbu69914OHDhA3759+eCDD7IdxP3NiBEjFJZFRKTYioiA4cNdDB/uIj4efvrJN+T7559tpKRkP1CfPWthypRgpkypx9ixJq1be+jSxU2XLm6aNfNgK5a/9Yj4kYgI3DExuGNiADCOHk1/ntq2dCmWkydzVazlyBEcX36J48svAfBcdRWeRo0ww8IgOBgzJAQzJARCQzEvfB8amnHfhb+DgtSrLQWu2P134vV6eeCBB/jqq6+44YYbeP3113NchmmaLFu2LP37HTt2cOrUKSpVqpSfVU13oQf6Ur3NCQkJRGgGQxEREcLDYehQF0OHukhKgp9/9g35/vFHO4mJ2f/FOC3NYPlyG8uX23jhBQgPN+nQwZ0eqhs00PJZInllVqmCa/hwXMOHg2li2bnz4vPUK1diJCTkqlzr9u1Yt2/PXZ0MI0PYvhCu078ODvYF8AtfXwjkl9h3oSwCAxXOJV2xCtEXeqCnTZvG0KFDczX82jRNRo8ezRdffMHgwYPp3bs3o0aNIiYmhtmzZxdIkL7wLPTevXtp2rRphn0nT54kMTGR5s2b5/t1RUREirOQEBgwwM2AAW5SU1NYtMjG99/bmTfPTnx8zn6ZjY83mDfPdy5AZKSXLl3cdO7sC9VRUX6/4qeIfzMMvA0a4GzQAOfIkeB2Y92wIb2X2rp2LYbLVfDVME1ISMh1gL8U02LJ2BNetiyeJk3wtGmDu1UrzGrVFLKvIMUmRP85QA8ePJj3338/x8OvTdPkkUceYfLkyQwaNIgPP/wQq9WKxWLhvvvuS5+1u2LFivla9w4dOvDaa6+xaNEihgwZkmHfwoUL048RERGRrAUGQp8+bvr0ceN0prB0qa+Heu5cG7//nvP5TE6etDBjhoMZMxwA1KrloWtXX6Du1MlDmTIK1SJ5YrPhad0aT+vWpI0dC0lJ2GJjLy6n9euvRV3DHDG8XoiPx7gwsnTvXmxr18KHHwLgjYzE07o17tat8bRqhadpU03MUIIVixB9YQj3tGnTGDhwYK6eXzZNkzFjxvDZZ59lCNAAQ4cOBcgQpCPzcUKDLl26UKNGDb7++mvuu+++9LWiz58/z2uvvYbD4cjxc90iIiJXKocDrrnGzTXXuHn9dVi50sr339uZM8fOqVO5myB03z4r+/ZZ+fjjAAzDpFEjb3qobtvWTUhIPt+EyJUmJAR3z564e/YEwDhzxjdJ2R/Dvy0HDxZxBfPGcvIklj9Nlmba7b6e6j8Fa7NKlSKupeQXIy4uzu8/ah0/fjz//e9/CQ0NZeTIkVkG6L59+6aH06wcO3aMLl260KFDBz766CNsWcwu8tVXXzFy5Ejeffddhg0bdtk6TZ48mdjYWAC2bdvG5s2badu2LTVr1gSgXbt23HrrrenHL1u2jCFDhhAYGMjgwYMJDQ1l1qxZHD58mOeff56HHnooWz+Lkiw1NZXDhw8TFRVFoD658wtqE/+i9vA/ahP/4vHAmjVu5sxJYfPmCvzyiz1HM31fit1u0qrVxZ7q5s092O35UOErhF4n/sVf28M4cCB9GSzb+vUYcXGQlIThdhd11fKNt2pV3K1a+Xqq27TB06gROBx+2yZyacUiRI8aNYqpU6de9pjsLOd04MABqlatmmWAvmDPnj3UqVMnz3W66aabmDhxYoZt69evZ/z48axduxaXy0V0dDQPPPAAgwcP/tvrXQn0BuJ/1Cb+Re3hf9Qm/ufPbWKagaxZY2XpUhtLltjYtMmKaeY9VIeGZpykLDpak5Rdjl4n/qXYtYfTiZGUBImJGMnJmb9OSsL440+mr5OTMRITfd//cbyRmOjb5yn6deXNwEA8TZuS1rw5x2vUoFTv3jiqVSvqakk2FIsQLVeGYvemfgVQm/gXtYf/UZv4n8u1SVwcLF9uY+lS35/du/Nnacvy5b3pE5R16eKmenX9avVnep34F7UHYJoXw/nlQngWXxtJSZCQgPW333K9pNfleKtXTx/+7W7dGm/DhhS79flMEyMuDuP4cSwnTlz8+8QJLMePYxw/juvmm3HecUdR1zTXilmLiIiIiORORATExLiJifENDz161GDZsouh+vjx3D1Pffq0hW++cfDNN75JymrU8NCliyd99u9y5RSqRfyKYUBAAGZAAJQpQ65eoaaJcegQtl9+wbpmDdZffsH666957uG2HDyI4+BB+Oor32WCg/E0b35xwrLWrTHLls3TNXLtj5nPswzGJ05g+dPXRlraZYvytG5dSJUuGArRIiIickWqUsXkpptc3HSTC9OE3bst6YF6+XIb58/nboz2gQNWDhyw8tlnvlDdsKEnvZe6fXs3oaH5eRciUiQMA7N6dVzVq+P6Y5JikpKwbtzoC9Zr12JduxbL2bN5u0xyMrYVK7CtWJG+zVO7dvpz1e5WrfBedRXkcNLlTJKTM4bjS4RkIykpb9f5g3HiRL6UU1QUokVEROSKZxhQr56XevWc3HOPE48HNm+2/hGqraxebSM1NXeheutWK1u3WpkwIQCbzTdJWYcObtq29dC6tZvw8Hy+GREpGiEheDp2xNOxo+9708Syf396oLatXYtl2zbfcll5YN27F+vevTBtmu8yYWG4W7RI76l2t2rlG3oDkJaGcfLk5XuPjx27uHRXIbEcP16o18tvCtEiIiIif2G1QvPmHpo39/Doo5CaCmvWWNOHf2/YYMXrzXmodrsNYmNtxMb6fgWzWEwaNvTStq2vl7ptWw8VK2r4t0iJYBh4a9XCW6sWrgvL2SYkYN2wAdvatb4h4GvXYomLy9tlEhKwL1mCfcmS9G3e6tUhMTHPPeEFRSFaREREpIQLDOSP55w9PPVUGnFxsHKlL1AvW2Zjx47cDaX0eg22bLGyZYuVDz4IAKBmTQ9t23po185N+/YeatfW7N8iJUZYGJ4uXfB06eL73uvF9dtvJPz0E5H79hGwYQOWHTswzLx9mOaP626bFgtmhQp4K1bErF69qKuTJwrRIiIiIjkUEQF9+7rp29c3Sdnx4xcnKVu2zMaRI7mbpAxg/34r+/dbmTrV90x1+fJe2rb1/NFb7aFRI0+xm6xXRC7BYsFTty5nAwMJjorCFRgIcXHY1q+/OAx8/fpCH26dU96yZTErVsRbubLv74oVMStVyvh3+fLFb6bxSygZdyEiIiJShCpVMhk2zMWwYb5Jyvbts6SvT718uZVz53Ifqk+ftjB7toXZs+0AhISYtGrlTu+tbtnSQ0hIft2JiBS5iAjcPXrg7tHD973Hg2XnzvTnqq2//IJ19+5CqYpZqhTeCyG4YkW8lSplDsmRkRAQUCj18RcK0SIiIiL5yDCgdm0vtWs7ufNOJ14vbNliSe+pXrPGRmJi7sdnJyUZLFliZ8kSX6i22UyaNPHQrp2vt7pdOw9ly+q5apESw2rFGx2NNzoa1+23A2D8/rsvTP/yC7Y1a7Bu2JCjmbPNkJCLgTirYFypEt7ISAgOLqCbKt4UokVEREQKkMUCTZt6adrUycMPO3G7YetWC6tX2/6YZMzKqVO576l2uw3Wr7exfr2Nd97x9QbVq/fnUO2menVTz1WLlCBmmTK4e/XC3asXaQBuN5Zt29KX17KcOIG3dOlLhmTCwtCbQu4pRIuIiIgUIpvtYqgeOdKJacL+/RZWrbL+Eayt7N2btzVfd+2ysmvXxbWqK1f2pvdSt23rJjram+dlZUXEj9hseBs3xtm4Mdx1V1HXpsRTiBYREREpQoYBtWp5qVXLy803uwA4dcogNvZiqN6yJXdLal1w7JiFb7918O23vu/Dw03atPGF6nbt3DRr5iEwMD/uRkSk5FOIFhEREfEzFSqYDBjgZsAA3+zfCQmwbp0tvbd63TorKSm5D9Xx8QYLFthZsMD3XLXDYdKixcVnqlu3dhMRkR93IiJS8ihEi4iIiPi5sDDo1s1Nt25uIA2nE7ZssRIba2XVKhurV+dtBnCn0/jj+Wwbr78OhmHSsKGX9u3df/zxUL68JisTEQGFaBEREZFix+GAli09tGzp4aGHfDOA79plYfXqC6HaxqFDuQ/Vpmnw669Wfv3VyvvvX5ys7EKgbt/eTdWqCtUicmVSiBYREREp5iwWaNDAS4MGXm6/3fdc9dGjRvoz1bGxNrZts2CauR8CfmGysk8/9X1frdrFnuoOHTzUquXVZL8ickVQiBYREREpgapUMRkyxMWQIb5QHRcHa9b4hn7HxtrYsMGK05n71HvokIVDhxxMm+abATwy0puhp/qqq7xYct8ZLiLitxSiRURERK4AERHQq5ebXr18z1WnpsKGDRdnAF+zxkZ8fO5D9cmTFmbOdDBz5oXref+YpMxCzZohVKyIZgAXkRJBIVpERETkChQYyB+9xh4APB747TcLq1f7ZgFftcrGqVO570qOi7Mwb56FefPswFUEB3tp08aT3lPdooWW1RKR4kkhWkRERESwWqFxYy+NGzu5914wTdi718KqVVZWrrSxapWNw4dzH6qTky0sXmxh8eKMy2p16OAbAt6qlZuwsPy6GxGRgqMQLSIiIiKZGAbUqeOlTh0vt97qe6760CHfUlirVvl6q3fvtua6/D8vqwVgtZo0aXKxp7pdOw+lS2sGcBHxPwrRIiIiIpIt1aqZVKvmYtgwX6g+dcogNvZiT/Vvv+V+BnCPx2DDBhsbNth45x3fslrR0Rd7qtu1c1OxokK1iBQ9hWgRERERyZUKFUwGDHAzYIAb8M0A7num2tdTvWmTFbc795OVbdtmZds2Kx9+6Pu+dm1fT3WHDm66dlWoFpGioRAtIiIiIvkiIgJ693bTu7cvVCcmwqpVXn78MYVt28qxYYOdtLTch+q9e63s3Wvl8899y2pFR3vo3t1N9+5u2rVzExSUH3chInJ5CtEiIiIiUiBCQ6FzZyc1ax4jKsqKYQSyYYP1j+eqfctrJSbmvaf6nXcCCAgwad/enR6qo6O9GLkvWkTkkhSiRURERKRQBARAu3Ye2rXzMGYMuN2wdasl/ZnqVausnDuXuxnA09IMFi+2s3ixnaeegshIL926+QJ1165uKlTQ0G8RyR8K0SIiIiJSJGw2aNrUS9OmTh54wInXCzt3WtID9cqVNk6cyF2oPnnSwrRpDqZN8w39btTowtBvF23beggIyM87EZEriUK0iIiIiPgFiwWuusrLVVc5uesu31rVBw5YWLnSmh6sDxzI3bJav/5q5ddfrbz5ZgBBQSYdO7rTe6rr19fQbxHJPoVoEREREfFLhgE1a3qpWdPLzTdfXKt6yRIbixbZWLLERlxcznuqU1IMFiyws2CBHYDKlX1Dv3v0cNOli5uyZTX0W0QuTSFaRERERIqNatVMbr3Vxa23uvB4YONGK4sW2Vi82MbatVY8npx3KR87ZmHKFAdTpjgwDJOmTX1Dv7t1c9O6tQeHowBuRESKLYVoERERESmWrFZo2dJDy5Yexo5N4/x5WLHCF6gXLbKxb1/Oh36bpsHGjTY2brTx6qsQEuIb+t29u6+nunZtDf0WudIpRIuIiIhIiVCqFPTt66ZvX9861QcOGCxaZGfRIhvLltmIj895+k1KMvjxRzs//ugb+h0V5U2foKxLFzcREfl5ByJSHChEi4iIiEiJVKOGyZ13OrnzTiduN6xf7xv6vWiRjfXrrXi9OQ/Vhw9b+OwzB5995sBiMWne3JM+QVnLlh7s9gK4ERHxKwrRIiIiIlLi2WzQpo2HNm08/POfacTFwdKlvqHfCxfaOXw45xOUeb0G69bZWLfOxssvQ3i4SadOvmepmzb10KCBh9DQ/L8XESlaCtEiIiIicsWJiIABA9wMGODGNFPZt8+S3ku9fLmNxMSc91LHxxvMnWtn7tyL3dHVq3uJjvZw9dUerrrK93WdOl71WIsUYwrRIiIiInJFMwyoXdtL7dpO7rnHidMJv/xiTZ+gbONGK6aZu9nEDh60cPCghXnzLqZmu92kbl1vhmB91VUeqlUzNWmZSDGgEC0iIiIi8icOB3To4KFDBw9PPpnG778bLF1qY+FC3/Dvo0dzPvT7z1wug23brGzblnH28LAwk6uuyhisr77aq3WrRfyMQrSIiIiIyGWUKWMyaJCLQYNcmCbs2mVJX5t6xQobycn5032ckGCwdq2NtWszbo+M9HLVVR6io73pwbp+fQ8hIflyWRHJIYVoEREREZFsMgyoX99L/fpORo1ykpYGa9ZcmPXbzpYtOV+b+u+cPGnh5EkLS5b8uR4mNWp4MwTrq67yPW9t02/4IgVKLzERERERkVwKCIDOnT107uzh2Wd9Q7+3brWkD9fevt3C9u3WXE1UdjmmabB/v5X9+60ZJjJzOEzq1fMNB/cNCfd9XbWqnrcWyS8K0SIiIiIi+aRMGTM9VF/g9cLhw0aGYL1tm5Vduyy43fmbbJ1Og61brWzdmrFHPDzc97x1/fp2IiPLc+21Nlq2RMFaJBcUokVERERECpDFAtWrm1Sv7ua669zp251O2LPHkh6sf/vNyvbtVg4ezNvEZVmJjzdYs8bGmjU2oDovvwxVqnjp3dvFdde56dTJTUBAvl9WpERSiBYRERERKQIOB0RH+55r/rOEBNixI2Ow3rbNwpkz+Ruujx61MGlSAJMmBRAaatK9u5vrrnNx7bVuzQguchkK0SIiIiIifiQsDFq18tCqlQdwpW8/fdpg27aMwXrHDitJSXkfk52YaDBrlp1Zs+xYLCZt2ni47jpfL3Xdut6/L0DkCqIQLSIiIiJSDJQvb9Kli4cuXTI+b33okJEhWG/fbmX37tw/b+31GsTG2oiNtfH001CnjofrrvP1Urdu7dHs33LF00tARERERKSYsligRg2TGjXc9O178XnrtDTYvduSHqwvTGp2+HDOh4Tv2WPl7betvP12AKVLe7n2Wjd9+rjo3t1NWFh+3o1I8aAQLSIiIiJSwgQEQMOGXho2zDgU+9SpNL7/Pp7166vw888BOX7O+tw5C9OnO5g+3YHDYdKpk2+ytN69XVStqueo5cqgEC0iIiIicoUIDzfp2jWOW24Jw24PZN06K/Pn25g3z86OHda/L+BPnE6DhQvtLFxo57HHgmjUyPccdZ8+Lpo08Wr5LCmxFKJFRERERK5AViu0aeOhTRsPzzyTxr59FubN8wXq2FgrHk/OUvCvv1r59Vcr//tfIJUr+5bP6t3bTefObgIDC+gmRIpA/i9CJyIiIiIixU6tWl4eeMDJnDlJ7NmTwIcfJjN4sJPw8JwP0z52zMLHHwdwww0h1K4dzs03BzNlip0zZ9Q9LcWfeqJFRERERCSD0qVNrr/exfXXu3A6U1i1ysoPP9iZN8+e48nJkpIM5syxM2eOHcPwLZ/Vu7dv+ax69TTsW4of9USLiIiIiMglORzQtauH//0vlS1bEli5MoEnn0ylRQv335/8F6ZpsHq1jWefDaJNmzBatAjlX/8KZMUKK+6cFydSJBSiRUREREQkWwwDrr7ay2OPpbFwYRI7dsTz5pvJ9OrlIjAw58O+9+2zMmFCAP36hVKnThj33hvEzJl24uMLoPIi+UTDuUVEREREJFcqVjS57TYXt93mIjkZlizxTUw2f76N06dz1l8XF2dhxgwHM2Y4sFpNmjf30LGjm06dPLRp4yYkpIBuQiSHFKJFRERERCTPgoOhTx83ffq48Xph/Xor8+bZmD/fzrZtOVs+y+Mx+OUXG7/8YuP118FmM2nR4kKodtO6tYfg4AK6EZG/oRAtIiIiIiL5ymKBVq08tGrl4emn0zhwwGDePN/EZCtX5nz5LLfbYM0aG2vW2Hj1VbDbTVq29NChw8VQHRRUQDcj8hcK0SIiIiIiUqBq1DAZNcrJqFFO4uLg55/tzJtnY8ECO/HxOZ+e2+UyiI21ERtr45VXwOHwheoLPdWtWnm0NrUUGIVoEREREREpNBERMHSoi6FDXbhcvuWzLvRSHzyYu3mPnU6DVatsrFpl43//g4AAk1atLobqli09BATk733IlUshWkREREREioTdDl26eOjSxcP48als325h4UIbK1b4AnFCQu4WkU5LM1ixwlfOSy9BYKBJ69YeOnVy07GjmxYtPDgc+XwzcsVQiBYRERERkSJnGBAd7SU62slDDzlxu2HLFisrVlhZvtw3dDsxMXehOjXVYNkyG8uW+eJPUJBJmza+mb87dnTTvLkHuz0/70ZKMoVoERERERHxOzYbNG/uoXlzDw8/7AvVmzZdDNWrV9tISspdqE5JMViyxM6SJb7kHBxs0rbtxVDdtKlCtVyaQrSIiIiIiPg9mw1atvTQsqWHRx5x4nLBxo1WVqywsXy5lTVrbCQn5y5UJycbLFpkZ9EiX3IODb0Qqt107OihSRMPNiUn+YP+KYiIiIiISLFjt0Pr1h5at/YwZgw4nbBhw4VQbWPNGiupqbkL1YmJBj//bOfnn32hOizMpF27i6G6cWMP1pwtfS0liEK0iIiIiIgUew4HtG3roW1bD489lkZaGqxffzFUr11rJS0td6E6IcHgp5/s/PSTL1SHh5u0bu2mTRsPbdr4JioLCcnPuxF/phAtIiIiIiIlTkAAtG/voX17D2PHppGaCuvW+Z6nXrHCxi+/WHE6cxeq4+Mz9lTbbCaNG3to08ZD27a+cF2xopmftyN+RCFaRERERERKvMBA6NjRQ8eOHiCNlBT45ZeLoXrdOisuV+5CtdttsGGDjQ0bbEyc6FuQukaNC6HaF6zr1/diyd0y2OJnFKJFREREROSKExQEnTt76NzZF6qTkzOHarc7d6Ea4MABKwcOWJk+3fd9qVK+ZbUu9FY3b+4hKCh/7kUKl0K0iIiIiIhc8YKDoUsXD126+EJ1UhKsXeub+XvFChsbNuQtVJ8/n/G5arvdpEkTDy1a2KhVK4I+fSxEReXTzUiBUogWERERERH5i5AQ6NbNTbdubiCNxERYs8bGihVWVq/2hercTlQG4HIZrFtnY906G1CHceOgVq2Lz1W3beuhbl0NAfdHCtEiIiIiIiJ/IzQUevRw06OHL1SnpcHmzVZWr/aF6jVrrJw9m7fEu2+flX37rEyd6gCgdGkvrVv7nqtu08Y3BDwwMB9uRvJEIVpERERERCSHAgIurlP98MNOTBP27rUQG2tlzRpfqN69O2+LSZ87Z+HHHy38+KNvCLjDYdK0qSd9aa22bT2UK6dZwAubQrSIiIiIiEgeGQbUqeOlTh0vt9ziAuDMGYM1ay6G6o0bc7+sFoDTabB2rY21a228/bZvFvA6dTKG6rp1vRi5v4Rkg0K0iIiIiIhIAShXzqRvXzd9+7oBSE2FTZsuDgFfu9bK77/nbQj4nj1W9uyxMmWKbwh42bK+IeCNG3to0MBL/foeatf2EhCQ59uRPyhEi4iIiIiIFILAQP5YN9oD+IaAb93qYv78RPbsiWTdOgd79+ZtCPjZsxbmzbMwb549fZvValKzppd69bw0aOChfn1fuK5b10tISB5v6gqkEC0iIiIiIlIEDAPq1vUQGHiGqKggAgMDOXUq4xDwTZusuFx5G5/t8RjpPdY//GDPsK9aNV+gvhCs69f3Uq+eh4iIPF2yRFOIFhERERER8RMVKpjExLiJifENAU9JgY0bL84AvmaNlbi4/Fv36tAhC4cOWViwIOP2SpV8Pdf16/uGhder5/tbE5kpRIuIiIiIiPitoCBo395D+/YeALxe2LXLkmFprf378zYEPCvHj1s4ftzC0qUZI2OZMl7q1/cNC/cND/cF7UqVzCtmQjOFaBERERERkWLCYoEGDXzh9fbbfbOAnzxpsHr1xSHgv/1mJTW1YBLt779biI21EBubMUqGh5vUq5dxWHj9+h6qVTOx5F/HuV9QiBYRERERESnGIiNNBgxwM2CAbwi4xwOHDxvs2GFl504LO3f6/t61y0pCQsGE6/h4g3XrbKxbl3F7UJBJ3bqZn7uuXduLNf870AuFQrSIiIiIiEgJYrVCjRomNWq46d374nbThGPHjPRQfSFg79hh4dy5gukuTkkx2LLFypYtGRPzgQPni+3kZQrRIiIiIiIiVwDDgCpVTKpUcdO9+8XtpglnzhgZeq0v/H3iRP6H60qVvMU2QINCtIiIiIiIyBXNMKB8eZPy5T107OjJsC8uDnbtsmYK2IcO5T5c16vnzWONi5ZCtIiIiIiIiGQpIgJat/bQurUHcKVvT0qC3bsz91zv32/B47n8c9f163suu9/fKUSLiIiIiIhIjoSEQNOmXpo2zdirnJYGe/daMj13vWePBafTF67r11dPtIiIiIiIiAgBARAd7SU6OmNQdrvhwAFfqG7YUD3RIiIiIiIiIpdks0GdOl7q1CnevdAAJWzZaxEREREREZGCoxAtIiIiIiIikk0K0SIiIiIiIiLZpBAtIiIiIiIikk0K0SIiIiIi8v/t3XtUVWX+x/E3iKYDKAYCyVW8pCRamKUwknjJFaQi2oUYbysNSR0dzXFqyp8VM2rZZY2pqZOmhpeoCEnHJFJhgJy0VBpSLhIqpjkaIJgCwu8PF2fmxO0cPXiBz2st1vI8+3me/eyz/e59vvsqIiZSEi0iIiIiIiJiIiXRIiIiIiIiIiZSEi23lFatWt3sIcivaJ3cWrQ+bj1aJ7cerZNbj9bJrUXr49ajdXJ7sSoqKqq+2YMQERERERERuR3oTLSIiIiIiIiIiZREi4iIiIiIiJhISbSIiIiIiIiIiZREi4iIiIiIiJhISbSIiIiIiIiIiZREi4iIiIiIiJhISbQ0qW+++YbHHnsMT09POnfuzLBhw4iPjzerj8uXL7NkyRL8/f1xcXGhZ8+ezJo1i7NnzzbRqJufU6dOsWLFCsaMGUPv3r3p1KkTPXr0YPz48ezfv9/kflJTU3FwcKj3LzY2tgmXovnx8/Or97sMDQ01q68PP/yQIUOG0LlzZ7y8vHjiiSc4ePBg0wy8mYqNjW3w/7eDgwOjRo1qtB/Fifm2bt3K7NmzGTx4MM7Ozo1+TyUlJbzwwgv07t0bZ2dn/Pz8eOmllygtLTV73snJyYSEhODu7o6HhwePPvooe/fuvZ7FaRZMXScVFRUkJCQwbdo0HnjgAdzc3HB3d2fo0KG89957XLlyxaz5WnK72NyYEyeLFi1qcDtUUFBg1rxzc3OZNGkSPj4+uLq6EhgYyHvvvUd1dct+U64566Sx/YuDgwMnT540ab6Kk5vP5mYPQJqvlJQUxo4dS9u2bQkPD8fOzo5t27YxefJkTp48ycyZMxvto6qqiqeeeork5GT69+/PqFGjyMvLY8OGDezdu5cvvvgCJyenG7A0t7fVq1fz9ttv06VLF4KDg3FyciIvL4/t27ezfft2/v73vxMeHm5yf4GBgfz2t7+tVe7n52fJYbcI7du3Jzo6ula5p6enyX0sXbqUmJgYPDw8mDx5MqWlpXzyySeMGDGChIQEBgwYYMkhN1t+fn7Mnz+/zmnbtm3j+++/Z+jQoSb3pzgxXUxMDCdOnMDR0REXFxdOnDhRb92ysjJCQ0PJzMxkyJAhjBs3jsOHD7Ns2TLS0tLYsWMHbdu2NWm+W7duJSoqCicnJyIiIgCIj48nLCyM999/n9GjR1tk+W5Hpq6T/Px8Jk6ciJ2dHUFBQTzyyCOUlJSwc+dO5s6dy65du9iyZQtWVlYmz9sS28XmyJw4qREREVHn99ahQweT53vkyBEefvhhLl26RFhYGHfddRe7du1i7ty5HDlyhNdff92s5WhOzFkn9e1f8vPz+fDDD+nZsyfu7u4mz1txcnNZFRUVtexDSNIkKisr6d+/P6dOnSIpKYk+ffoAUFxczNChQzl+/Dj79+9vNNA/+OADZsyYwbhx41izZo1hJ7x27VrmzJnDpEmTePvtt5t6cW5727Zt484776z1gz49PZ3Ro0dja2vL0aNHueOOOxrsJzU1lZEjRzJ//nyef/75phxyi1CTTGVmZl5zH3l5eTz44IN4e3uTnJxs+GF0+PBhhg8fjre3NxkZGVhb68Kja1VeXk7Pnj0pKSkhKysLZ2fnBusrTsy3Z88efHx88PT05K233uLll19m+fLlREZG1qr717/+lddee43Zs2ezcOFCQ/nChQt5++23WbBgAXPmzGl0nkVFRfTt2xcbGxtSUlJwc3MDoLCwkKCgIAAOHjyIvb29ZRbyNmPqOjl16hQ7duwgIiICW1tbQ3lZWRmPPvoo3377Le+//z5hYWEmzdcS28Xmypw4WbRoEUuWLCExMZFBgwZd13xDQkJIT08nLi6O4cOHA1e3i6NHjyYjI4Ndu3bxwAMPXNc8blfmrJP6zJs3jzVr1hATE8OMGTNMaqM4ufn0q0qaREpKCvn5+YwbN86QQMPVI59z5syhvLyczZs3N9rPhg0bAFiwYIHRUezJkyfj7e1NXFwcv/zyi+UXoJkZNWpUnWfEAgICGDRoEEVFRWRlZd2Ekcn1io2NpbKykrlz5xqdWejTpw9jx47l6NGjZGRk3MQR3v62b9/O+fPnGTFiRKMJtFybwYMHm3T2pLq6mo0bN2JnZ8e8efOMps2bNw87OzvDfqMxn376KcXFxTzzzDOGBBrAzc2NqVOncu7cOT777DPzFqQZMXWddO7cmSlTphgl0AC2trZMnz4dgLS0tCYZY0tj6jqxpNzcXNLT0xk0aJAhgQZo06YNf/7znwFYv379DR3TreR618mlS5eIi4ujTZs2PPnkkxYcmTQ1Xc4tTeKf//wnAEOGDKk1reZyyMZ2qpcuXWL//v1079691gbKysqK4OBg1q1bx7fffktAQICFRt7ytG7dGoBWrVqZ3ObYsWOsWLGCS5cu0blzZ4KCgujcuXNTDbFZKy8vJzY2ltOnT2Nvb4+/vz/333+/ye0bi7VNmzaRlpZGYGCgxcbc0tQkZRMmTDCrneLE8vLy8vjxxx8ZOnRonUnbgw8+SHJyMidPnmz0ssjGYmfx4sWkpaUZLvMW813L/gWuf7so/5Wens6BAwewtrbGx8eHwYMHY2dnZ3L7huJk4MCB2Nra6iDJdUhMTKSoqIjRo0ebfXui4uTmUhItTSIvLw+Arl271prm4uKCnZ0dx44da7CP/Px8qqqq8PHxqXN6TXleXp6S6Gt04sQJ9uzZg6urK/fcc4/J7eLi4oiLizN8trGx4ZlnnuHVV181+8dSS3fmzBnD2Zoa/v7+vPfee3Tp0qXR9nl5edjZ2eHi4lJrWk381cSjmO/48ePs3bsXNzc3hg0bZlZbxYnl1fxfbmi/kJycTF5eXqNJdEP7KcWOZXzwwQdA3QlYQ653uyj/tWjRIqPPHTp0YPHixSYfHGoo5lq1aoWXlxdHjhyhsrISGxulFebauHEjYP5BWlCc3Gy6nFuaRElJCXD1oQd1sbe3N9RprI/6Hn5R03dj/UjdKioqiIqK4vLlyyxcuNCkH/VOTk4sXLiQjIwMCgsLycnJITY2Fh8fH1asWMGCBQtuwMibj8jISBISEsjJyeHUqVOkpKTwxBNP8M033zBq1CguXLjQaB8lJSUNxllNHbk2sbGxVFVVERERYXLiqzhpOpbcLzS0n1LsXL/333+fpKQkgoKCePjhh01uZ4ntokDv3r155513OHjwIKdPn+bQoUO89tprWFlZ8eyzz7Jjxw6T+mks5uzt7amqqrqmJ+O3dD/88AOpqam4u7sTHBxsVlvFyc2nQ0YiLVBVVRXPPvss6enpTJw40eT7cHr16kWvXr0Mn21tbQkNDeX+++8nMDCQVatWMXv2bDp16tRUQ29W/vSnPxl97tOnD6tWrQKuPjV4/fr1Jj9kRCyvqqqK2NhYrKys+N3vfmdyO8WJtHQ7d+5k3rx5eHh4sHr1arPaartoGSNHjjT67OXlxTPPPMPdd99NWFgYMTExhISE3KTRCVy9UqO6uprIyEizH/6pOLn5dCZamkRjZwMuXLhQ79mzX/dRXFxc5/TGznZL3aqqqpg+fTpxcXE8/vjjvPXWW9fdp4uLCyEhIVRWVpr13mmp2+TJkwHYt29fo3Xbt2/fYJzV1BHz7dmzh5MnTxIUFIS3t/d196c4uX6W3C80tJ9S7Fy7Xbt2MXHiRJydnUlMTMTV1dUi/ZqzXZT6PfTQQ3Tp0oWsrCyTrrRoLOYuXLiAlZWVWfdZy9XfYps3b8ba2tqsg7SNUZzcOEqipUk0dD/ZmTNnKC0trfeethre3t5YW1vXe+90TXld97NJ3WrOQG/evJlx48axcuVKi736yNHREYCLFy9apL+WzJzvsmvXrpSWlnLmzJla0xq651Mad60PFGuI4uT61PxftsR+oaH9lGLn2nz++eeMHz8eR0dHEhMTLXLwqYZix3JqvktT3m7SUMxduXKFgoICvLy8dD+0mb744gsKCwsJDg7Gw8PDYv0qTm4cJdHSJGqeBPzll1/WmpacnGxUpz7t2rWjX79+5OTkcPz4caNp1dXV7N69G1tbW+677z4Ljbp5q0mgt2zZQnh4OKtWrbLow41qzqzd6NdvNEfmfJeWiDWp7fz58+zYsYOOHTvy6KOPWqxfxcn16dq1K3fddRf79u2jrKzMaFpZWRn79u3Dy8ur0YeKgWLH0j7//HMmTJhAx44dSUxMbPRAubkUO5ZRVlbGkSNHsLW1NSRcDWkoTjIyMigrK1OcXIPreaBYQxQnN46SaGkSDz30EN7e3nz00UccPnzYUF5cXMybb75Z6314p0+fJjs7u9blQhMnTgTglVdeobq62lC+bt06fvjhBx577DHatWvXxEtz+6u5hHvLli2EhYWxevXqBhPoc+fOkZ2dzblz54zKDx48WGf9lStXkpqaSteuXfH397fk0Jut7OzsOo8UZ2dns3DhQgDGjRtnKC8uLiY7O5vTp08b1Y+MjMTGxoY33njDKH4OHz7Mxx9/zN13383AgQObZiGasS1btlBeXs7jjz/OHXfcUWcdxcmNZ2Vlxfjx4yktLeX11183mvb6669TWlpq2G/UuHjxItnZ2Zw4ccKofMyYMbRv357Vq1dTWFhoKC8sLGTNmjU4Ojpa9ABKc5aUlMSECRNwcHAgMTGx0TP4FRUVZGdnk5+fb1Ru7nZR6nbhwgVyc3Nrlf/yyy/MmjWLCxcuEBYWVuvscXZ2NtnZ2UZl3bt3JyAggNTUVJKSkgzl5eXl/OUvfwEsnwg2d//5z3/YuXMnTk5OPPLII/XWU5zc2qyKioqqG68mYr6UlBTGjh1L27ZtCQ8Px87Ojm3btnHixAleffVVZs6caagbHR3N5s2bWb58OZGRkYbyqqoqHnvsMZKTk+nfvz+BgYEcO3aMxMREPD09SU5ONvu9ei3RokWLWLJkCXZ2dkybNq3OBDo0NJQ+ffoY1Z8/fz7PP/+8oY6fnx+tW7fmvvvuo3Pnzly8eJGvv/6aw4cP06FDBz755BP69et3w5brdrZo0SJWrFhBQEAAHh4e/OY3vyE3N5ekpCQqKiqYM2eO0VOcY2NjmT59OhEREaxcudKor6VLlxITE4OHhwejRo2itLSUTz75hPLychISEhgwYMCNXrzbXkBAAFlZWaSlpdX7+jfFieVs2LCBjIwMALKysjh06BADBgwwvKZl4MCBhh/qZWVljBgxgu+++44hQ4bQt29fDh06xJdffom/vz/bt283OriamprKyJEjCQwMZPv27Ubz3bp1K1FRUTg5OTFmzBgA4uPjOXfuHOvWrSMsLOwGLP2tydR1kp2dzaBBg7h8+TJjx46lW7dutfry9PQ02rcXFBTQt29fPDw8yMzMNJSbu11saUxdJwUFBdx77734+/vTo0cPXFxc+Omnn9i7dy+FhYX4+vry2Wefceeddxr17+DgAEBRUZFR+ffff8+IESO4dOkSY8aMwdXVlV27dvH9998zderUWge0WhJztl01li1bxksvvcT06dMNByLqoji5tekGBmkyQUFB7Ny5k0WLFhEfH09FRQW+vr68/PLLhIeHm9SHtbU1mzZt4q233mLr1q2sWLGCjh07Mn78eF588UUl0CaquRy+tLSUpUuX1lnH09PTkETX5+mnnyY5OZn09HTOnz+PtbU1Hh4eREdHM2PGDNzc3Cw+9uZq0KBBZGdnc/jwYTIyMrh48SKOjo4MHz6cKVOmmPVe1eeeew5PT09WrlzJ2rVrad26NQMHDuSFF17g3nvvbbqFaKYOHDhAVlYW/fr1M+v96TUUJ+bLyMhg8+bNRmVfffUVX331leFzzQ9RW1tbtm/fzuLFi0lMTCQ1NRUXFxdmzJjB/Pnzzbo66YknnsDR0ZE33niDTZs2YWVlRd++fZk3bx6DBw+2yLLdrkxdJ2fOnOHy5csAfPzxx3X2FRgYaJRE18eS28XmyNR10rFjR6ZMmcKBAwdISkqiqKiIdu3a0aNHD6Kiopg6dapZcdKrVy+Sk5OJiYlh165dXLx4ka5du7J06VKefvppiy3f7cicbVeNmvenX+sZfMXJrUFnokVERERERERMpHuiRUREREREREykJFpERERERETEREqiRUREREREREykJFpERERERETEREqiRUREREREREykJFpERERERETEREqiRUREREREREykJFpERERERETEREqiRUREREREREykJFpERERuWwUFBTg4OODg4HCzhyIiIi2Ezc0egIiIiDSt0NBQ0tLSTKpbVFTUtIMRERG5zSmJFhERaSHc3d1xd3e/2cMQERG5rSmJFhERaSEiIyN5/vnnb/YwREREbmu6J1pERERERETEREqiRUREpJZfP7DrH//4B6GhoXh5eeHm5sawYcP48MMPG+zj6NGjTJ8+HT8/P5ydnfHy8iIkJIQNGzZw5cqVettVVlayadMmwsPD6datG87OzvTq1YuQkBCWLVtGcXFxvW0zMjJ4/PHH6dKlC66urgQEBLB69Wqqq6vrrH/w4EGmTp1K7969cXZ2xs3NDT8/P8aOHcuyZcvqbSciIi2XLucWERGRBq1atYr58+fTsWNHfHx8KCwsZP/+/Ya/1157rVab+Ph4oqKiKC8vx9bWFl9fX37++WfS09NJT08nPj6eTZs20a5dO6N2586d46mnnmLfvn0AuLi40Lt3b86ePcu+fftIT0/n3nvvZdCgQbXmGRsby8yZM+nQoQPe3t6cOHGCrKws/vjHP3L8+HFiYmKM6n/xxRdERERQUVGBnZ0d3bp1w8bGhlOnTpGcnExycjLR0dHY2OjnkoiI/JfORIuIiEiDXnzxRebNm0dOTg67d+/m6NGjvPnmm1hbW7N69Wo+/fRTo/rZ2dlER0dTXl7OhAkTyM7OZs+ePRw6dIhPP/2U9u3bs3v3bhYsWGDUrrq6mokTJ7Jv3z7c3d1JSEjg6NGjfPnll2RmZnLs2DHeeOMNnJ2d6xznnDlziImJITc3l927d5Obm2uYx/Lly8nPzzeqv3DhQioqKpg1axY5OTmkp6eTkpJCbm4umZmZvPzyy1hb66eSiIgYsyoqKtJ1SiIiIs2Yqa+4CgkJYdOmTcDVy7n79u0LQHBwMPHx8bXqz5o1i/Xr1+Pr60t6erqhfMaMGXzwwQf4+vqSlpaGlZWVUbsNGzbw+9//ntatW5OZmYmrqytw9ZLxiIgI7rjjDlJSUrj77rsbHfP/jnP8+PEsW7asVp2AgACysrJYvHgx06ZNM5S7uLhw+fJlCgoK6NChQ6PzEhERAZ2JFhERaTHc3d0ZMGBAvX89e/ass110dHSD5VlZWZw8edJQnpSUBMC0adNqJdAAERERdOrUiYqKCnbv3m0o37ZtGwAjR440KYH+tSlTptRZ/sADDwBw7Ngxo3IPDw8APvroI7PnJSIiLZdu8hEREWkhrvUVV7169aqzvHv37tjY2FBZWUl2djbu7u4UFxdz5swZAHx9fets17p1a7p3787Zs2fJyckxlGdlZQH/TXrN1a1btzrLO3XqBEBpaalR+axZs5g5cyZz587lnXfeITg4mP79+xMYGIinp+c1jUFERJo/nYkWERGRBtV3D3KrVq248847Abhw4QJgnKjW1w4wXMJd0+5//32tl1bb2trWWV5zX/Ovn7Q9fvx4Nm7cyIMPPkhBQQFr164lOjqaPn36MGzYMFJTU69pHCIi0rwpiRYREZEG/fTTT3WWX7lyhfPnzwNgb28PgJ2dXaPtAE6fPm3U7n//3dArrCxt5MiRfP755+Tn5xMXF8cf/vAHunTpwv79+xk7diyZmZk3bCwiInJ7UBItIiIiDTpy5Eid5Tk5OVRWVgLQo0cP4OpZZBcXF+C/l2f/WmVlpeEy7pp2APfccw8A//rXvywzcDN06NCB4cOH83//9398/fXX9O/fn/LycjZs2HDDxyIiIrc2JdEiIiLSoHfffbfBcl9fX9zd3Q3lDz/8sGH6ry+hBtiyZQtnz56ldevWBAcHG8pHjx4NQGJiotG90jeajY0N/fr1A+DHH3+8aeMQEZFbk5JoERERaVBKSgpLliwxnHWurq5m/fr1bNy4EYDnnnvOqP7MmTNp27YtWVlZzJ49m7KyMsO0vXv38uKLLwIwadIkw1lrgBEjRhAUFMTly5cJDw+vdU9ySUkJa9eu5ejRo9e9TCUlJUycOJHk5GTKy8uNph08eNDwSi9/f//rnpeIiDQvejq3iIhICxEbG8vevXsbrLNkyRLDe5drxMTEMH/+fN599126dOlCYWGh4QncU6ZMITw83Kh+jx49WLlyJVFRUaxfv56PP/6Y7t278/PPP/PDDz8AV989/corr9Sa/7p163jyySf5+uuvGTlyJK6urri5uXH27FkKCwu5cuUKiYmJ1/QKrP9VVVVFQkICCQkJtGnTBh8fH2xtbTl79izHjx8H4P777zd6r7SIiAgoiRYREWkxTp48afQ+57qUlJTUKouKisLDw4Ply5eTmZlJZWUl/fr1Y+rUqTz55JN19jNmzBh69erF3/72N1JSUvj3v/9N27ZtGThwIBEREURGRtKqVata7RwdHdmxYwebN28mLi6O7777jszMTJycnBgwYAChoaG1kvxrYW9vz5o1a0hJSeHAgQOcPn2a4uJi7O3tGThwIGPGjGHSpEm0adPmuuclIiLNi1VRUVHtm5VERESkRSsoKDAkq0VFRTd3MCIiIrcQ3RMtIiIiIiIiYiIl0SIiIiIiIiImUhItIiIiIiIiYiIl0SIiIiIiIiIm0oPFREREREREREykM9EiIiIiIiIiJlISLSIiIiIiImIiJdEiIiIiIiIiJlISLSIiIiIiImIiJdEiIiIiIiIiJlISLSIiIiIiImIiJdEiIiIiIiIiJlISLSIiIiIiImIiJdEiIiIiIiIiJvp/nkHhkqH3UqYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = sbs_doc_emb.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1ZpLkF80PIpj",
    "outputId": "21052d89-b2a7-4095-f299-1121da84d61c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[437, 447],\n",
       "        [291, 324]])"
      ]
     },
     "execution_count": 164,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_doc_emb.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kLKZdrC9PIpj"
   },
   "source": [
    "# BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PtgTF0CIPIpk",
    "outputId": "4d39d1ff-63a0-4fa3-b2c3-adb393cb7cd7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.bert.modeling_bert.BertModel'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModel\n",
    "auto_model = AutoModel.from_pretrained('bert-base-uncased')\n",
    "print(auto_model.__class__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "id": "bYjuuJG-PIpk"
   },
   "outputs": [],
   "source": [
    "from transformers import BertModel\n",
    "bert_model = BertModel.from_pretrained('bert-base-uncased')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IpDTWGNBPIpk",
    "outputId": "325ec284-3ec9-4bc3-ebe3-0355bd011cb2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertConfig {\n",
       "  \"_name_or_path\": \"bert-base-uncased\",\n",
       "  \"architectures\": [\n",
       "    \"BertForMaskedLM\"\n",
       "  ],\n",
       "  \"attention_probs_dropout_prob\": 0.1,\n",
       "  \"classifier_dropout\": null,\n",
       "  \"gradient_checkpointing\": false,\n",
       "  \"hidden_act\": \"gelu\",\n",
       "  \"hidden_dropout_prob\": 0.1,\n",
       "  \"hidden_size\": 768,\n",
       "  \"initializer_range\": 0.02,\n",
       "  \"intermediate_size\": 3072,\n",
       "  \"layer_norm_eps\": 1e-12,\n",
       "  \"max_position_embeddings\": 512,\n",
       "  \"model_type\": \"bert\",\n",
       "  \"num_attention_heads\": 12,\n",
       "  \"num_hidden_layers\": 12,\n",
       "  \"pad_token_id\": 0,\n",
       "  \"position_embedding_type\": \"absolute\",\n",
       "  \"transformers_version\": \"4.32.0\",\n",
       "  \"type_vocab_size\": 2,\n",
       "  \"use_cache\": true,\n",
       "  \"vocab_size\": 30522\n",
       "}"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fBkvBjp5PIpk"
   },
   "source": [
    "## Tokenization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "M7ZcsRmdPIpk",
    "outputId": "19550fb1-08d8-46e1-8993-b8ae3577459d",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30522"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import BertTokenizer\n",
    "bert_tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",
    "len(bert_tokenizer.vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kyF47gBWPIpk",
    "outputId": "00c5dbec-5f49-401e-878c-cac075ad67e5"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  2003,  1999, 10288, 24759,  5555,  6321,  2206,  1996,\n",
       "          2317, 10442,   102,  3582,  1996,  2317, 10442,  1010,  9253,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'Alice is inexplicably following the white rabbit'\n",
    "sentence2 = 'Follow the white rabbit, Neo'\n",
    "tokens = bert_tokenizer(sentence1, sentence2, return_tensors='pt')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "brcF_Ox1PIpl",
    "outputId": "d4caa8d8-1042-488a-b348-57568df3c7d6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '##ica', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]', 'follow', 'the', 'white', 'rabbit', ',', 'neo', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(bert_tokenizer.convert_ids_to_tokens(tokens['input_ids'][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1Cjrk1qCPIpl",
    "outputId": "aded8752-3c8d-4c5b-b40f-bfd54e8574ed"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.bert.tokenization_bert_fast.BertTokenizerFast'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "auto_tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')\n",
    "print(auto_tokenizer.__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j0A5jpZJPIpl"
   },
   "source": [
    "## Input Embeddings\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_input_embed.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JOhc49UmPIpl",
    "outputId": "3bd4c3da-cdd3-4574-9b20-3fd787e19549"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertEmbeddings(\n",
       "  (word_embeddings): Embedding(30522, 768, padding_idx=0)\n",
       "  (position_embeddings): Embedding(512, 768)\n",
       "  (token_type_embeddings): Embedding(2, 768)\n",
       "  (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)\n",
       "  (dropout): Dropout(p=0.1, inplace=False)\n",
       ")"
      ]
     },
     "execution_count": 172,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_embeddings = bert_model.embeddings\n",
    "input_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Zo5g1KN2PIpm",
    "outputId": "da1a5f33-0604-4a0d-b3c2-e27f32af0e80"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(30522, 768, padding_idx=0)"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "token_embeddings = input_embeddings.word_embeddings\n",
    "token_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jr_sA4KEPIpm",
    "outputId": "9def548b-e0e9-41b1-f6ec-7311aca8365a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.3630e-02, -2.6490e-02, -2.3503e-02,  ...,  8.6805e-03,\n",
       "           7.1340e-03,  1.5147e-02],\n",
       "         [-6.9710e-02, -8.8202e-02,  5.0619e-03,  ...,  1.4105e-02,\n",
       "           2.1815e-02, -1.3769e-02],\n",
       "         [-3.6044e-02, -2.4606e-02, -2.5735e-02,  ...,  3.3691e-03,\n",
       "          -1.8300e-03,  2.6855e-02],\n",
       "         ...,\n",
       "         [ 5.2089e-05, -1.0468e-02, -9.9103e-03,  ...,  1.4558e-02,\n",
       "           1.3217e-02,  2.2406e-02],\n",
       "         [-3.5037e-02, -7.2933e-02, -3.6124e-02,  ..., -5.7723e-02,\n",
       "          -5.5074e-03,  7.2688e-03],\n",
       "         [-1.4521e-02, -9.9615e-03,  6.0263e-03,  ..., -2.5035e-02,\n",
       "           4.6379e-03, -1.5378e-03]]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 174,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_token_emb = token_embeddings(tokens['input_ids'])\n",
    "input_token_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rx5LiAACPIpm",
    "outputId": "166fa35b-5aa6-4647-a6bb-b02c2d37752c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(512, 768)"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "position_embeddings = input_embeddings.position_embeddings\n",
    "position_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "W6e0kmEnPIpm",
    "outputId": "feb5ced9-f1d0-403d-f043-d400259fe2c1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n",
       "          14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,\n",
       "          28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,\n",
       "          42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,\n",
       "          56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,\n",
       "          70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,\n",
       "          84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,\n",
       "          98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,\n",
       "         112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,\n",
       "         126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,\n",
       "         140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,\n",
       "         154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,\n",
       "         168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,\n",
       "         182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,\n",
       "         196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,\n",
       "         210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,\n",
       "         224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,\n",
       "         238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,\n",
       "         252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,\n",
       "         266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,\n",
       "         280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,\n",
       "         294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,\n",
       "         308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,\n",
       "         322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,\n",
       "         336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,\n",
       "         350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,\n",
       "         364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,\n",
       "         378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,\n",
       "         392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,\n",
       "         406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,\n",
       "         420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,\n",
       "         434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,\n",
       "         448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,\n",
       "         462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,\n",
       "         476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,\n",
       "         490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,\n",
       "         504, 505, 506, 507, 508, 509, 510, 511]])"
      ]
     },
     "execution_count": 176,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "position_ids = torch.arange(512).expand((1, -1))\n",
    "position_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SqZUNJBhPIpn",
    "outputId": "e9488fae-f267-4a27-f6a0-b5c82bb52665"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.7505e-02, -2.5631e-02, -3.6642e-02,  ...,  3.3437e-05,\n",
       "           6.8312e-04,  1.5441e-02],\n",
       "         [ 7.7580e-03,  2.2613e-03, -1.9444e-02,  ...,  2.8910e-02,\n",
       "           2.9753e-02, -5.3247e-03],\n",
       "         [-1.1287e-02, -1.9644e-03, -1.1573e-02,  ...,  1.4908e-02,\n",
       "           1.8741e-02, -7.3140e-03],\n",
       "         ...,\n",
       "         [-9.2809e-03,  8.3268e-03, -4.1643e-03,  ...,  3.4903e-02,\n",
       "          -1.8319e-02, -2.9017e-03],\n",
       "         [-8.5999e-03,  3.2205e-04, -2.1249e-03,  ...,  2.7744e-02,\n",
       "          -7.2760e-03, -2.0280e-03],\n",
       "         [-3.4622e-04, -8.3709e-04, -2.2228e-02,  ...,  2.3493e-02,\n",
       "          -4.5198e-04, -5.7741e-04]]], grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seq_length = tokens['input_ids'].size(1)\n",
    "input_pos_emb = position_embeddings(position_ids[:, :seq_length])\n",
    "input_pos_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5DXX4yiEPIpn",
    "outputId": "72c96dc2-9048-4b58-eb98-0b3af745beca"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(2, 768)"
      ]
     },
     "execution_count": 178,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "segment_embeddings = input_embeddings.token_type_embeddings\n",
    "segment_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "GgCRoV2sPIpn",
    "outputId": "c9e136f6-0558-417d-9504-3d81b6b78e81"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         [ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         [ 0.0004,  0.0110,  0.0037,  ..., -0.0066, -0.0034, -0.0086],\n",
       "         ...,\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112],\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112],\n",
       "         [ 0.0011, -0.0030, -0.0032,  ...,  0.0047, -0.0052, -0.0112]]],\n",
       "       grad_fn=<EmbeddingBackward0>)"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_seg_emb = segment_embeddings(tokens['token_type_ids'])\n",
    "input_seg_emb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iUkjZrKTPIpn",
    "outputId": "291cb1c4-f3b3-44fc-981b-daaee4add8fc"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 0.0316, -0.0411, -0.0564,  ...,  0.0021,  0.0044,  0.0219],\n",
       "         [-0.0615, -0.0750, -0.0107,  ...,  0.0364,  0.0482, -0.0277],\n",
       "         [-0.0469, -0.0156, -0.0336,  ...,  0.0117,  0.0135,  0.0109],\n",
       "         ...,\n",
       "         [-0.0081, -0.0051, -0.0172,  ...,  0.0542, -0.0103,  0.0083],\n",
       "         [-0.0425, -0.0756, -0.0414,  ..., -0.0252, -0.0180, -0.0060],\n",
       "         [-0.0138, -0.0138, -0.0194,  ...,  0.0032, -0.0011, -0.0133]]],\n",
       "       grad_fn=<AddBackward0>)"
      ]
     },
     "execution_count": 180,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_emb = input_token_emb + input_pos_emb + input_seg_emb\n",
    "input_emb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dGkuZ7UdPIpn"
   },
   "source": [
    "## Pretraining Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m2FoG2AGPIpn"
   },
   "source": [
    "### Masked Language Model (MLM)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_mlm.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZTw0eDKePIpo",
    "outputId": "51618919-0318-4647-c158-a0dc5d7cfa98"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[101, 5650, 2003, 1999, 10288, 24759, 5555, 6321, 2206, 1996, 2317, 10442, 102]"
      ]
     },
     "execution_count": 181,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'Alice is inexplicably following the white rabbit'\n",
    "tokens = bert_tokenizer(sentence)\n",
    "tokens['input_ids']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3yCbwZXgPIpo",
    "outputId": "9a0e3565-4f14-43a1-9d06-a78b396207c4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  2003,  1999, 10288, 24759,   103,  6321,  2206,  1996,\n",
       "          2317, 10442,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]), 'labels': tensor([[-100, -100, -100, -100, -100, -100, 5555, -100, -100, -100, -100, -100,\n",
       "         -100]])}"
      ]
     },
     "execution_count": 182,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import DataCollatorForLanguageModeling\n",
    "torch.manual_seed(41)\n",
    "data_collator = DataCollatorForLanguageModeling(tokenizer=bert_tokenizer, mlm_probability=0.15)\n",
    "mlm_tokens = data_collator([tokens])\n",
    "mlm_tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XI180xPwPIpo",
    "outputId": "e66ce80c-e224-4e3e-e618-6b7d2c5f45bd",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '##ica', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]']\n",
      "['[CLS]', 'alice', 'is', 'in', '##ex', '##pl', '[MASK]', '##bly', 'following', 'the', 'white', 'rabbit', '[SEP]']\n"
     ]
    }
   ],
   "source": [
    "print(bert_tokenizer.convert_ids_to_tokens(tokens['input_ids']))\n",
    "print(bert_tokenizer.convert_ids_to_tokens(mlm_tokens['input_ids'][0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gyTICtkbPIpo"
   },
   "source": [
    "### Next Sentence Prediction (NSP)\n",
    "\n",
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_nsp.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eWrQN9DpPIpo",
    "outputId": "ddff94bb-6d99-4608-d4d8-b7c97db6c2f2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BertPooler(\n",
       "  (dense): Linear(in_features=768, out_features=768, bias=True)\n",
       "  (activation): Tanh()\n",
       ")"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.pooler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WmVOtEwePIpp",
    "outputId": "5cf022ac-aa77-400f-c7ea-ac8abebc3bd3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  5650,  4076,  1996,  2317, 10442,   102,  3582,  1996,  2317,\n",
       "         10442,  9253,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 185,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence1 = 'alice follows the white rabbit'\n",
    "sentence2 = 'follow the white rabbit neo'\n",
    "bert_tokenizer(sentence1, sentence2, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_HkUpyLUPIpp"
   },
   "source": [
    "## Outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "5C-AaHkFPIpp",
    "outputId": "360fa058-8ed6-4043-b0cf-b6ff4a5ee55d"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'When he had finished, he thanked her kindly and oiled himself thoroughly with his jeweled oil-can, to guard against mishap.'"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = train_dataset[0]['sentence']\n",
    "sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AmqPWjJuPIpp",
    "outputId": "3bce9a26-6a76-4456-cb99-435b019ab393"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2043,  2002,  2018,  2736,  1010,  2002, 15583,  2014, 19045,\n",
       "          1998,  3514,  2098,  2370, 12246,  2007,  2010, 13713,  2098,  3514,\n",
       "          1011,  2064,  1010,  2000,  3457,  2114, 28616,  3270,  2361,   102]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "         0, 0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "         1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 187,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = bert_tokenizer(sentence,\n",
    "                        padding='max_length',\n",
    "                        max_length=30,\n",
    "                        truncation=True,\n",
    "                        return_tensors=\"pt\")\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f_ssmrA7PIpp",
    "outputId": "89b73566-52f2-4011-88fb-351de9ff96ca"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "odict_keys(['last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'])"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bert_model.eval()\n",
    "out = bert_model(input_ids=tokens['input_ids'],\n",
    "                 attention_mask=tokens['attention_mask'],\n",
    "                 output_attentions=True,\n",
    "                 output_hidden_states=True,\n",
    "                 return_dict=True)\n",
    "out.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n0lNYShtPIpq"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/bert_embeddings.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O9qGa0qaPIpq",
    "outputId": "76d39e78-b2c8-4ff2-9e3b-8a50390f91d9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-4.8754e-01, -1.2611e-01,  3.3714e-01,  ..., -4.5496e-03,\n",
       "         -7.1811e-02,  1.9374e-01],\n",
       "        [ 4.0133e-04,  4.1986e-01,  3.5203e-01,  ...,  2.4569e-01,\n",
       "          5.3264e-01, -2.8016e-01],\n",
       "        [-1.5727e-02,  1.5502e-01, -4.6268e-01,  ...,  3.1876e-01,\n",
       "          2.9846e-01,  2.7099e-01],\n",
       "        ...,\n",
       "        [-8.1689e-01, -4.3263e-01,  6.3697e-01,  ..., -8.7997e-02,\n",
       "          3.8065e-02, -6.4068e-01],\n",
       "        [-5.1033e-01, -1.5985e-01,  2.8810e-01,  ..., -6.8723e-01,\n",
       "          8.2599e-02, -2.4723e-01],\n",
       "        [-4.9477e-01, -4.2874e-01, -2.2155e-01,  ...,  4.9540e-01,\n",
       "          1.2636e-02, -6.2742e-01]], grad_fn=<SliceBackward0>)"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "last_hidden_batch = out['last_hidden_state']\n",
    "last_hidden_sentence = last_hidden_batch[0]\n",
    "# Removes hidden states for [PAD] tokens using the mask\n",
    "mask = tokens['attention_mask'].squeeze().bool()\n",
    "embeddings = last_hidden_sentence[mask]\n",
    "# Removes embeddings for the first [CLS] and last [SEP] tokens\n",
    "embeddings[1:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rlqC7lDSPIpq",
    "outputId": "3223bb39-f6b1-4a89-ffa4-c1abc75fc991"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.5084, -0.1625,  0.2651,  ...,  0.0074, -0.0110,  0.2201],\n",
       "        [ 0.0993,  0.4398,  0.2474,  ...,  0.3561,  0.5003, -0.2802],\n",
       "        [-0.0191,  0.0916, -0.4948,  ...,  0.4507,  0.2689,  0.4180],\n",
       "        ...,\n",
       "        [-0.9531, -0.2325, -0.3269,  ..., -0.4476,  0.4718,  0.1520],\n",
       "        [-0.7924, -0.4712,  0.5397,  ..., -0.0063, -0.0828, -0.7425],\n",
       "        [ 0.8010,  0.2180, -0.3146,  ...,  0.4345, -0.5236, -0.2188]],\n",
       "       device='cuda:0')"
      ]
     },
     "execution_count": 190,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_embeddings(bert_flair, sentence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hNKAkRLyPIpq",
    "outputId": "60dd09c1-69a9-4539-a6ee-48198ff98909"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n",
      "torch.Size([1, 30, 768])\n"
     ]
    }
   ],
   "source": [
    "print(len(out['hidden_states']))\n",
    "print(out['hidden_states'][0].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "451QgYohPIpq",
    "outputId": "60a2c571-1140-427d-8a83-d1b33e198702"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['hidden_states'][0] == bert_model.embeddings(tokens['input_ids'])).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "93o1_w-EPIpr",
    "outputId": "21087261-9be8-458f-8687-b984bfc61ad9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['hidden_states'][-1] == out['last_hidden_state']).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bYA0p0XqPIpr",
    "outputId": "e8eabe1d-c27e-478c-c143-a21b2d7fe5f3"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(True)"
      ]
     },
     "execution_count": 194,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(out['pooler_output'] == bert_model.pooler(out['last_hidden_state'])).all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jsdP4CnSPIpr",
    "outputId": "13a87eec-336d-4cc6-ef76-b318b11e3c3e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12\n",
      "torch.Size([1, 12, 30, 30])\n"
     ]
    }
   ],
   "source": [
    "print(len(out['attentions']))\n",
    "print(out['attentions'][0].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PlX9GImRPIpr"
   },
   "source": [
    "## Model IV - Classifying using BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {
    "id": "zWB0IsJEPIpr"
   },
   "outputs": [],
   "source": [
    "class BERTClassifier(nn.Module):\n",
    "    def __init__(self, bert_model, ff_units, n_outputs, dropout=0.3):\n",
    "        super().__init__()\n",
    "        self.d_model = bert_model.config.dim\n",
    "        self.n_outputs = n_outputs\n",
    "        self.encoder = bert_model\n",
    "        self.mlp = nn.Sequential(\n",
    "            nn.Linear(self.d_model, ff_units),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(dropout),\n",
    "            nn.Linear(ff_units, n_outputs)\n",
    "        )\n",
    "\n",
    "    def encode(self, source, source_mask=None):\n",
    "        states = self.encoder(input_ids=source,\n",
    "                              attention_mask=source_mask)[0]\n",
    "        cls_state = states[:, 0]\n",
    "        return cls_state\n",
    "\n",
    "    def forward(self, X):\n",
    "        source_mask = (X > 0)\n",
    "        # Featurizer\n",
    "        cls_state = self.encode(X, source_mask)\n",
    "        # Classifier\n",
    "        out = self.mlp(cls_state)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rN86j0y9PIps"
   },
   "source": [
    "### Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {
    "id": "LLFm2rAtPIps"
   },
   "outputs": [],
   "source": [
    "def tokenize_dataset(hf_dataset, sentence_field, label_field, tokenizer, **kwargs):\n",
    "    sentences = hf_dataset[sentence_field]\n",
    "    token_ids = tokenizer(sentences, return_tensors='pt', **kwargs)['input_ids']\n",
    "    labels = torch.as_tensor(hf_dataset[label_field])\n",
    "    dataset = TensorDataset(token_ids, labels)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 145,
     "referenced_widgets": [
      "c68e3cfe691a4e17bff5dd0d6fa645d3",
      "deea6050951d4ca58e70d9a690dbbed9",
      "f2cbe349777d4fb28e1d02ade5f6f63a",
      "d5224726057743ee8d441a0827240ea8",
      "85f7913cfdb24be095daf7d1610192c6",
      "a42616c094cb4485920db4bb2972b8e8",
      "b6cd1e7ed31d468baa7e9a322ea4b36a",
      "344401e1b0234ea394d535dfc2f02da4",
      "017e198210e54a189160e6c5448b45ff",
      "533e80c06ed342e98f4caf25ec21f03b",
      "4978f765d5a346ec94880e2a3c1dc73d",
      "c5fe4d01057141068e6bb47ace62576d",
      "e7d783b2428a4ee487dc4e700e4ed9fc",
      "704fabe6625d498ea91de8237e8bc600",
      "63cb88869a0e473dacec5a6e43f4731a",
      "e9cd1445c89746eeacc43611394f05e6",
      "09c09951186c4b8baca1824cece58586",
      "6f7870b5ddab4cd6ae6d6df571d624b0",
      "7987211ff4eb48a2a042ccc78a57b964",
      "5bbcad5e25f74579b2ab8d64e3b77723",
      "58c6a66e017447d990f0383c1fdb57f0",
      "bf91c89808db4dd498cfd7da6dda0cec",
      "b1b0bd4a30364fb79620dc284fb591fb",
      "30653f3a5d36426e970064e40ede2418",
      "f37ddc958e61436795033581348ee8ff",
      "518537d8640346c4bb9544e5011fda3c",
      "e93a503d473b411dbf651736f2697704",
      "09c64d67093745c1aab40091ab7e80d6",
      "511c36e2cdcd4eaeb6db3dd6af3e14c8",
      "020e13cca93f4c20a5df7384b61b56d8",
      "5c9d700dd2ec4c0d92231652d9d50758",
      "8a6ff1cc71c24b37afbdf4e532978c63",
      "88ebe60388384796910b1987ca2a7b36",
      "1e5982c69228437689fee07d4b252fab",
      "fd480f04a0354a7ab1ac9fd4d531ccc1",
      "805b82526c7c4cb48a1c3dc795b7153f",
      "fa05df95f5b540859234430c7a9f8a92",
      "53c46000a715472c813fe4138446826f",
      "e7803472ac394d368b137951385998ed",
      "6ff4ff5036844be9bbd0993eb6173596",
      "bff1c8d9ebb64b728c6fc43deff520f9",
      "08d599a00ddb4b348bf3818fd86b098f",
      "fcbc067f13494ff2b9798686ab0a8547",
      "dc2d8ab4dab447a5bfca5f7df1db3b4f"
     ]
    },
    "id": "hvCj__6mPIps",
    "outputId": "57c5a8c9-cef6-411f-a126-c21e3895ada1"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c68e3cfe691a4e17bff5dd0d6fa645d3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)okenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c5fe4d01057141068e6bb47ace62576d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/483 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b1b0bd4a30364fb79620dc284fb591fb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1e5982c69228437689fee07d4b252fab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')\n",
    "tokenizer_kwargs = dict(truncation=True, padding=True, max_length=30, add_special_tokens=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "bc52309682db4f1daac4df480b481dd4",
      "ef6585ad9f58426486fe68e416a5085b",
      "a07ca3c194074fc382bce90671da91dc",
      "4dacc2dfaec841b9b09643d32cfb142f",
      "014cde879c6b4bf486dbd88bb4f15aba",
      "36f064f978d34560ad0536674d9115cd",
      "74566debe4c24ceabfe4867202dc998f",
      "5cc744a17a254e739b953518f2b4d47c",
      "f59beaa8010b4e4fa173ae0e4bb0ebbd",
      "f9459b5a73dc44d58a68925f08598e65",
      "cb2fea0bc72b4a8486f09d387442624b",
      "c920c622ebca4d4484ae524879220748",
      "5af30da4716c4bfc96a7ff53ab2255f1",
      "b8207f4f24304030b4d5944afa13137d",
      "2db3d91800bd40078fd85d53ea88a042",
      "a8f94a7dcd194cee8c44f683dd80b9d8",
      "d17523ae711f442690b81f29b351d363",
      "a03055e17979435892821dcb79ae1524",
      "d22fe83a00414a7db643fc242c0767ac",
      "70c71e159df441f1819bc5b1ce8be618",
      "2f7c6c45c3354158a526792d8c8324f5",
      "aa2e85f8edcc44e3a83828950c0ee1ea"
     ]
    },
    "id": "bRuvrZRfPIps",
    "outputId": "09203bef-1128-4862-dc28-1068507c610a"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bc52309682db4f1daac4df480b481dd4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c920c622ebca4d4484ae524879220748",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_dataset_float = train_dataset.map(lambda row: {'labels': [float(row['labels'])]})\n",
    "test_dataset_float = test_dataset.map(lambda row: {'labels': [float(row['labels'])]})\n",
    "\n",
    "train_tensor_dataset = tokenize_dataset(train_dataset_float, 'sentence', 'labels', auto_tokenizer, **tokenizer_kwargs)\n",
    "test_tensor_dataset = tokenize_dataset(test_dataset_float, 'sentence', 'labels', auto_tokenizer, **tokenizer_kwargs)\n",
    "\n",
    "generator = torch.Generator()\n",
    "train_loader = DataLoader(train_tensor_dataset, batch_size=4, shuffle=True, generator=generator)\n",
    "test_loader = DataLoader(test_tensor_dataset, batch_size=8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a_zRWx9gPIps"
   },
   "source": [
    "### Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "a19cf0f8087a4b77b6f63e687f4c66eb",
      "2df66cf412d4416aa4d4120ef82f283d",
      "d3b257e5f68f493b9426b9169f8201be",
      "2689aebe8b564cd7b1f174c8b2adcb3e",
      "3f81f775fb0c478b8423db9bddbfad2e",
      "3dcd50ad000b4860bc1fd8f4566d50e4",
      "be84a6c67cec43ad81a2defc92698ffd",
      "446fed872cbf44e59688454c1fdbebb8",
      "e3147854e3e849cd8badb22bca577f83",
      "d2e070514ba74de3ac0ed266d311a1a0",
      "684d4d25c7654a24b7e50775fd8f5bc4"
     ]
    },
    "id": "3Z1oEHNkPIps",
    "outputId": "4cfdc8bf-40a1-4334-d735-27211f8921f6"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a19cf0f8087a4b77b6f63e687f4c66eb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "torch.manual_seed(41)\n",
    "bert_model = AutoModel.from_pretrained(\"distilbert-base-uncased\")\n",
    "model = BERTClassifier(bert_model, 128, n_outputs=1)\n",
    "loss_fn = nn.BCEWithLogitsLoss()\n",
    "optimizer = optim.Adam(model.parameters(), lr=1e-5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {
    "id": "yZL4UQsVPIpt"
   },
   "outputs": [],
   "source": [
    "sbs_bert = StepByStep(model, loss_fn, optimizer)\n",
    "sbs_bert.set_loaders(train_loader, test_loader)\n",
    "sbs_bert.train(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4Uma227MPIpt",
    "outputId": "1a70e5aa-d62a-41e9-a9b0-3ee67bee8b47"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "66461441"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sbs_bert.count_parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AOE-3A4jPIpt",
    "outputId": "ddd80e59-3666-4451-b2d6-7678f786255f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[438, 447],\n",
       "        [310, 324]])"
      ]
     },
     "execution_count": 203,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "StepByStep.loader_apply(test_loader, sbs_bert.correct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M-YD9bvDPIpt"
   },
   "source": [
    "# Fine-Tuning with HuggingFace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BXihCcZSPIpt"
   },
   "source": [
    "## Sequence Classification (or Regression)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {
    "id": "VsEGVD3wPIpt"
   },
   "outputs": [],
   "source": [
    "from transformers import DistilBertForSequenceClassification\n",
    "torch.manual_seed(42)\n",
    "bert_cls = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "w7MQ1ApTPIpu",
    "outputId": "31804e0c-1628-4e73-c758-01408cbceb2e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.distilbert.modeling_distilbert.DistilBertForSequenceClassification'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModelForSequenceClassification\n",
    "auto_cls = AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased', num_labels=2)\n",
    "print(auto_cls.__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_3eWB1c2PIpu"
   },
   "source": [
    "## Tokenized Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {
    "id": "VmJOKQcmPIpu"
   },
   "outputs": [],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')\n",
    "def tokenize(row):\n",
    "    return auto_tokenizer(row['sentence'],\n",
    "                          truncation=True,\n",
    "                          padding='max_length',\n",
    "                          max_length=30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "f1315b0496244e24addeb478d9e64eb8",
      "fd4997c1fe6743218f8bfcf5cfc8f1bd",
      "14a84284b58c4927a57600bd63b83f60",
      "7ae512c0087942209b5c83c124bf0eb9",
      "e5063a3999bd4629ab6ab8ce92dc647f",
      "4f8ef38fd5614431a7a3c974a7583570",
      "3f6619d00b9c4d4ca573b54d6f59d012",
      "169b92244a9e42cf89ed5d7b6fe643a7",
      "89b32c1717c342b28edc5c90be607f0c",
      "0dcb056b573242769c81edf55126e236",
      "2e7a9176007c49b8b6b623b3679fcac0",
      "6d1dc17b66e54c768c9ccd75f91d3e91",
      "826781bbc9f24c7e8cf7d3c7dfb00d01",
      "631233cb0483400eb95bb77b4217f7c4",
      "620aa9d515854ad181db286a7fb44ae8",
      "ac23cca0aca54d70bd0c13fca43386d2",
      "a8ca1717ef74473b99d901a76c0028c9",
      "1628cb8828844f46b2c3d673117e0831",
      "a3d8dcdcb68140cc9a836824d0b301b9",
      "b00f91de164a49c3bfe1759a685a126f",
      "e9905466f9744164971e84d9bbb0354c",
      "18d184687008423c94560b773dcf41f9"
     ]
    },
    "id": "FzW2xAeDPIpu",
    "outputId": "84386066-6c74-443a-de2e-3a74684ab6e7"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f1315b0496244e24addeb478d9e64eb8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3081 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d1dc17b66e54c768c9ccd75f91d3e91",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/771 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_train_dataset = train_dataset.map(tokenize, batched=True)\n",
    "tokenized_test_dataset = test_dataset.map(tokenize, batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MCToy9BpPIpv",
    "outputId": "88230106-865e-4d5c-ec9f-773e3b4a51e1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'sentence': 'When he had finished, he thanked her kindly and oiled himself thoroughly with his jeweled oil-can, to guard against mishap.', 'source': 'wizoz10-1740.txt', 'labels': 0, 'input_ids': [101, 2043, 2002, 2018, 2736, 1010, 2002, 15583, 2014, 19045, 1998, 3514, 2098, 2370, 12246, 2007, 2010, 13713, 2098, 3514, 1011, 2064, 1010, 2000, 3457, 2114, 28616, 3270, 2361, 102], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}\n"
     ]
    }
   ],
   "source": [
    "print(tokenized_train_dataset[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {
    "id": "OJGHlBQ9PIpv"
   },
   "outputs": [],
   "source": [
    "tokenized_train_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])\n",
    "tokenized_test_dataset.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nsQJ3GjfPIpv",
    "outputId": "104046c5-0805-4871-80b0-084766968cdf"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'labels': tensor(0),\n",
       " 'input_ids': tensor([  101,  2043,  2002,  2018,  2736,  1010,  2002, 15583,  2014, 19045,\n",
       "          1998,  3514,  2098,  2370, 12246,  2007,  2010, 13713,  2098,  3514,\n",
       "          1011,  2064,  1010,  2000,  3457,  2114, 28616,  3270,  2361,   102]),\n",
       " 'attention_mask': tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "         1, 1, 1, 1, 1, 1])}"
      ]
     },
     "execution_count": 210,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_train_dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pWBcmaN1PIpv"
   },
   "source": [
    "## Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {
    "id": "dHy5s29tPIpv"
   },
   "outputs": [],
   "source": [
    "from transformers import Trainer\n",
    "trainer = Trainer(model=bert_cls, train_dataset=tokenized_train_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O7pDhbwdPIpw",
    "outputId": "e3f6e199-f225-4d82-aa39-280d19c42857"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TrainingArguments(\n",
       "_n_gpu=1,\n",
       "adafactor=False,\n",
       "adam_beta1=0.9,\n",
       "adam_beta2=0.999,\n",
       "adam_epsilon=1e-08,\n",
       "auto_find_batch_size=False,\n",
       "bf16=False,\n",
       "bf16_full_eval=False,\n",
       "data_seed=None,\n",
       "dataloader_drop_last=False,\n",
       "dataloader_num_workers=0,\n",
       "dataloader_pin_memory=True,\n",
       "ddp_backend=None,\n",
       "ddp_broadcast_buffers=None,\n",
       "ddp_bucket_cap_mb=None,\n",
       "ddp_find_unused_parameters=None,\n",
       "ddp_timeout=1800,\n",
       "debug=[],\n",
       "deepspeed=None,\n",
       "disable_tqdm=True,\n",
       "dispatch_batches=None,\n",
       "do_eval=False,\n",
       "do_predict=False,\n",
       "do_train=False,\n",
       "eval_accumulation_steps=None,\n",
       "eval_delay=0,\n",
       "eval_steps=None,\n",
       "evaluation_strategy=no,\n",
       "fp16=False,\n",
       "fp16_backend=auto,\n",
       "fp16_full_eval=False,\n",
       "fp16_opt_level=O1,\n",
       "fsdp=[],\n",
       "fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_grad_ckpt': False},\n",
       "fsdp_min_num_params=0,\n",
       "fsdp_transformer_layer_cls_to_wrap=None,\n",
       "full_determinism=False,\n",
       "gradient_accumulation_steps=1,\n",
       "gradient_checkpointing=False,\n",
       "greater_is_better=None,\n",
       "group_by_length=False,\n",
       "half_precision_backend=auto,\n",
       "hub_always_push=False,\n",
       "hub_model_id=None,\n",
       "hub_private_repo=False,\n",
       "hub_strategy=every_save,\n",
       "hub_token=<HUB_TOKEN>,\n",
       "ignore_data_skip=False,\n",
       "include_inputs_for_metrics=False,\n",
       "jit_mode_eval=False,\n",
       "label_names=None,\n",
       "label_smoothing_factor=0.0,\n",
       "learning_rate=5e-05,\n",
       "length_column_name=length,\n",
       "load_best_model_at_end=False,\n",
       "local_rank=0,\n",
       "log_level=passive,\n",
       "log_level_replica=warning,\n",
       "log_on_each_node=True,\n",
       "logging_dir=tmp_trainer/runs/Aug27_14-25-59_c5bed8f1b093,\n",
       "logging_first_step=False,\n",
       "logging_nan_inf_filter=True,\n",
       "logging_steps=500,\n",
       "logging_strategy=steps,\n",
       "lr_scheduler_type=linear,\n",
       "max_grad_norm=1.0,\n",
       "max_steps=-1,\n",
       "metric_for_best_model=None,\n",
       "mp_parameters=,\n",
       "no_cuda=False,\n",
       "num_train_epochs=3.0,\n",
       "optim=adamw_torch,\n",
       "optim_args=None,\n",
       "output_dir=tmp_trainer,\n",
       "overwrite_output_dir=False,\n",
       "past_index=-1,\n",
       "per_device_eval_batch_size=8,\n",
       "per_device_train_batch_size=8,\n",
       "prediction_loss_only=False,\n",
       "push_to_hub=False,\n",
       "push_to_hub_model_id=None,\n",
       "push_to_hub_organization=None,\n",
       "push_to_hub_token=<PUSH_TO_HUB_TOKEN>,\n",
       "ray_scope=last,\n",
       "remove_unused_columns=True,\n",
       "report_to=['tensorboard'],\n",
       "resume_from_checkpoint=None,\n",
       "run_name=tmp_trainer,\n",
       "save_on_each_node=False,\n",
       "save_safetensors=False,\n",
       "save_steps=500,\n",
       "save_strategy=steps,\n",
       "save_total_limit=None,\n",
       "seed=42,\n",
       "sharded_ddp=[],\n",
       "skip_memory_metrics=True,\n",
       "tf32=None,\n",
       "torch_compile=False,\n",
       "torch_compile_backend=None,\n",
       "torch_compile_mode=None,\n",
       "torchdynamo=None,\n",
       "tpu_metrics_debug=False,\n",
       "tpu_num_cores=None,\n",
       "use_cpu=False,\n",
       "use_ipex=False,\n",
       "use_legacy_prediction_loop=False,\n",
       "use_mps_device=False,\n",
       "warmup_ratio=0.0,\n",
       "warmup_steps=0,\n",
       "weight_decay=0.0,\n",
       ")"
      ]
     },
     "execution_count": 212,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.args"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {
    "id": "vMVuTOSUPIpw"
   },
   "outputs": [],
   "source": [
    "from transformers import TrainingArguments\n",
    "training_args = TrainingArguments(\n",
    "    output_dir='output',\n",
    "    num_train_epochs=1,\n",
    "    per_device_train_batch_size=1,\n",
    "    per_device_eval_batch_size=8,\n",
    "    evaluation_strategy='steps',\n",
    "    eval_steps=300,\n",
    "    logging_steps=300,\n",
    "    gradient_accumulation_steps=8,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {
    "id": "LFxIhoybPIpw"
   },
   "outputs": [],
   "source": [
    "def compute_metrics(eval_pred):\n",
    "    predictions = eval_pred.predictions\n",
    "    labels = eval_pred.label_ids\n",
    "    predictions = np.argmax(predictions, axis=1)\n",
    "    return {\"accuracy\": (predictions == labels).mean()}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {
    "id": "XKo42k7RPIpw"
   },
   "outputs": [],
   "source": [
    "trainer = Trainer(model=bert_cls,\n",
    "                  args=training_args,\n",
    "                  train_dataset=tokenized_train_dataset,\n",
    "                  eval_dataset=tokenized_test_dataset,\n",
    "                  compute_metrics=compute_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9wdDiBP4PIpw",
    "outputId": "3c0366f2-c3cd-4c6f-d944-110073e8e1d3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 0.1832, 'learning_rate': 1.103896103896104e-05, 'epoch': 0.78}\n",
      "{'eval_loss': 0.15004514157772064, 'eval_accuracy': 0.9571984435797666, 'eval_runtime': 0.8666, 'eval_samples_per_second': 889.666, 'eval_steps_per_second': 111.929, 'epoch': 0.78}\n",
      "{'train_runtime': 83.6395, 'train_samples_per_second': 36.837, 'train_steps_per_second': 4.603, 'train_loss': 0.18022566708651455, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=385, training_loss=0.18022566708651455, metrics={'train_runtime': 83.6395, 'train_samples_per_second': 36.837, 'train_steps_per_second': 4.603, 'train_loss': 0.18022566708651455, 'epoch': 1.0})"
      ]
     },
     "execution_count": 216,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "j_i6kvF5PIpx",
    "outputId": "ca5bdb0c-3b38-4781-cdcb-b8186a69bd73"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 0.12053246051073074, 'eval_accuracy': 0.9688715953307393, 'eval_runtime': 1.2524, 'eval_samples_per_second': 615.63, 'eval_steps_per_second': 77.453, 'epoch': 1.0}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 0.12053246051073074,\n",
       " 'eval_accuracy': 0.9688715953307393,\n",
       " 'eval_runtime': 1.2524,\n",
       " 'eval_samples_per_second': 615.63,\n",
       " 'eval_steps_per_second': 77.453,\n",
       " 'epoch': 1.0}"
      ]
     },
     "execution_count": 217,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2KOMV6iQPIpx",
    "outputId": "77f79632-0ff2-45c8-cdbd-eb032ca31c2c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['training_args.bin', 'pytorch_model.bin', 'config.json']"
      ]
     },
     "execution_count": 218,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.save_model('bert_alice_vs_wizard')\n",
    "os.listdir('bert_alice_vs_wizard')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1WpIt7WvPIpx",
    "outputId": "63ea9de2-7bb0-4a4b-e1eb-d97b16863f97"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 219,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model = AutoModelForSequenceClassification.from_pretrained('bert_alice_vs_wizard')\n",
    "loaded_model.device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LrmSH5kSPIpx",
    "outputId": "cb09dfb3-bd24-499f-fe1b-8e203703d91a",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda', index=0)"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "loaded_model.to(device)\n",
    "loaded_model.device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I_6MnEPTPIpx"
   },
   "source": [
    "## Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3_dt7C28PIpy",
    "outputId": "e86833d8-d866-4d7f-dde0-75df47ee9458"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2091,  1996,  3756,  5318, 10442,  4920,   102]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentence = 'Down the yellow brick rabbit hole'\n",
    "tokens = auto_tokenizer(sentence, return_tensors='pt')\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WAJtVv5fPIpy",
    "outputId": "f938576d-b639-41a9-aca7-09ce25e3d2e2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.tokenization_utils_base.BatchEncoding'>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[  101,  2091,  1996,  3756,  5318, 10442,  4920,   102]],\n",
       "       device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]], device='cuda:0')}"
      ]
     },
     "execution_count": 222,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(type(tokens))\n",
    "tokens.to(loaded_model.device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Fs4-3g2DPIpy",
    "outputId": "2cf45084-1163-45ff-9c5e-df554f8a4ee6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SequenceClassifierOutput(loss=None, logits=tensor([[ 2.3088, -2.5816]], device='cuda:0', grad_fn=<AddmmBackward0>), hidden_states=None, attentions=None)"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_model.eval()\n",
    "logits = loaded_model(input_ids=tokens['input_ids'], attention_mask=tokens['attention_mask'])\n",
    "logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_81JJd68PIpy",
    "outputId": "dd03f7c2-427c-49e0-d969-aee1370301c1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0], device='cuda:0')"
      ]
     },
     "execution_count": 224,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logits.logits.argmax(dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AOIE2XY-PIpy"
   },
   "source": [
    "## Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {
    "id": "eTdPgNWAPIpy"
   },
   "outputs": [],
   "source": [
    "from transformers import TextClassificationPipeline\n",
    "device_index = loaded_model.device.index if loaded_model.device.type != 'cpu' else -1\n",
    "classifier = TextClassificationPipeline(model=loaded_model,\n",
    "                                        tokenizer=auto_tokenizer,\n",
    "                                        device=device_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rkhu6QaOPIpz",
    "outputId": "fca40e1e-7559-47ec-ecb2-9656af2f1b95"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'LABEL_0', 'score': 0.9925376772880554},\n",
       " {'label': 'LABEL_1', 'score': 0.9985089898109436}]"
      ]
     },
     "execution_count": 226,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier(['Down the Yellow Brick Rabbit Hole', 'Alice rules!'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {
    "id": "57PmBUrZPIpz"
   },
   "outputs": [],
   "source": [
    "loaded_model.config.id2label = {0: 'Wizard', 1: 'Alice'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2xsmzplHPIpz",
    "outputId": "fca40f55-da16-4772-89bc-5713a12e830c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'label': 'Wizard', 'score': 0.9925376772880554},\n",
       " {'label': 'Alice', 'score': 0.9985089898109436}]"
      ]
     },
     "execution_count": 228,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier(['Down the Yellow Brick Rabbit Hole', 'Alice rules!'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MPv9QGnPPIpz"
   },
   "source": [
    "## More Pipelines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 145,
     "referenced_widgets": [
      "917fb99088eb4329894cef75e45023ca",
      "590434dd89ee48e29e4a8a2c274c5c53",
      "04bd756693f14b21bd2901fc2e73d3ef",
      "94196f53645a4731b78a215090acc5c6",
      "69782579b2ad412b992f2204b9325dd8",
      "4ab2a7cacbd4448bb7e1900139ba7f1a",
      "4341486e91614581909fa67fadc90d48",
      "c0ffdf9164164d5c9213c7dd10116f23",
      "4ab5ac2fbe27424db741224634932854",
      "3991e93f6d5d4f8e8eee2363a34e224c",
      "ad73151124ce47fcb81a3c73718296a4",
      "2bf8da4a7a5341999ef9f95ac5c176c9",
      "5e35c165cbb14ba0a5f3098b362a6ef6",
      "79947dd0ddf34e5192a2678b52533a97",
      "078f35a0fcb8401e9e3b39366ccdfa22",
      "671a21b847754ec8b027956e53c5f337",
      "670ba37f36524d7699ff3b5be9952b09",
      "1a6934c3c9614c45a6ab239dcf90bc6a",
      "0bffcdb38d1d4f3c85e93210aabfa404",
      "7ebd8e2c1fcc429896627463d2319bab",
      "84904b49bcfb4f70a267c906a4981493",
      "931213e9c0ee47a799a98c5a0a57a764",
      "89856f13c9d247acb31b7802b1cca970",
      "b143e584661a42969b6a9f376546486a",
      "47b17d821f924db3a592d734f77ce092",
      "544015661b65451eb7bc40178d156eb7",
      "d3d5424dca584e38ad28fa4a32fc618b",
      "8ac0286de75940438cdaee591a0d3d24",
      "6c7078294d0a48d1b628e396e704497d",
      "15a667e533e945c894af026b818db869",
      "598a61bb19da461c9257b86a21c963cf",
      "3fbaf05579a345c3b2ec0e1fd0698c19",
      "20b0a8dbb55b4458ad16371a5f9617ec",
      "52712fe41859413aae6f6b75af710b90",
      "462794da68e542f381bff271e24185fa",
      "44b2d5e007b84f85b4f183ba0631500d",
      "7a15f0aba34e4c3e90bfbe605fef9bb5",
      "919e99b706034ed6aa81a1e903cfaf73",
      "e791c7e96e2a4ffdb8f6bdb12e61aa02",
      "6756718ae9e14f23858d2d0000d11fce",
      "e311df68b62b44099c1269883678c94b",
      "d792c590bc9f4fa18d66294f4c840ce9",
      "a000821e9b994a63964db53c2ffae61c",
      "c2fb075140864489a7d81c19a0f79e21"
     ]
    },
    "id": "orvV3n9wPIpz",
    "outputId": "a377a931-92aa-4601-a447-fd11a6c8746d"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "917fb99088eb4329894cef75e45023ca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/629 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2bf8da4a7a5341999ef9f95ac5c176c9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "89856f13c9d247acb31b7802b1cca970",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)okenizer_config.json:   0%|          | 0.00/48.0 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "52712fe41859413aae6f6b75af710b90",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "sentiment = pipeline('sentiment-analysis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ZGiwPvGfPIp0",
    "outputId": "0ec9692c-cdff-4ee9-a3af-67aaf78231b3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When he had finished, he thanked her kindly and oiled himself thoroughly with his jeweled oil-can, to guard against mishap.\n",
      "[{'label': 'POSITIVE', 'score': 0.9805866479873657}]\n"
     ]
    }
   ],
   "source": [
    "sentence = train_dataset[0]['sentence']\n",
    "print(sentence)\n",
    "print(sentiment(sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "x456QWjEPIp0",
    "outputId": "a05c3ccd-9ba5-49c0-d711-1e0d7a7188d4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'impl': transformers.pipelines.text_classification.TextClassificationPipeline,\n",
       " 'tf': (transformers.models.auto.modeling_tf_auto.TFAutoModelForSequenceClassification,),\n",
       " 'pt': (transformers.models.auto.modeling_auto.AutoModelForSequenceClassification,),\n",
       " 'default': {'model': {'pt': ('distilbert-base-uncased-finetuned-sst-2-english',\n",
       "    'af0f99b'),\n",
       "   'tf': ('distilbert-base-uncased-finetuned-sst-2-english', 'af0f99b')}},\n",
       " 'type': 'text'}"
      ]
     },
     "execution_count": 232,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers.pipelines import SUPPORTED_TASKS\n",
    "# UPDATED\n",
    "###########################################################\n",
    "# sentiment-analysis was replaced by text-classification\n",
    "# in the dictionary of supported tasks\n",
    "# SUPPORTED_TASKS['sentiment-analysis']\n",
    "SUPPORTED_TASKS['text-classification']\n",
    "###########################################################"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Xk3TWVXxPIp0",
    "outputId": "fce49265-5ef5-428a-d21b-24a3f018edc8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'impl': transformers.pipelines.text_generation.TextGenerationPipeline,\n",
       " 'tf': (transformers.models.auto.modeling_tf_auto.TFAutoModelForCausalLM,),\n",
       " 'pt': (transformers.models.auto.modeling_auto.AutoModelForCausalLM,),\n",
       " 'default': {'model': {'pt': ('gpt2', '6c0e608'), 'tf': ('gpt2', '6c0e608')}},\n",
       " 'type': 'text'}"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SUPPORTED_TASKS['text-generation']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qpfW8LWvPIp0"
   },
   "source": [
    "# GPT-2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 209,
     "referenced_widgets": [
      "6b436b11235a41c98bf37edcab77e6a8",
      "028efd79c4aa49f090fde0d01507518e",
      "2f84f62aa77b4738b3eff4bacfe28179",
      "c7789d7688fd44159f8680d23616d9ec",
      "db7fe2b698aa417fa37b850a5b9ff9ad",
      "77cc41136f654cc289f578ffe99b47f9",
      "d229ed8924f44660afd51f751a62074b",
      "bebba38c2ec54dcfaaf2097d3dd3f4c3",
      "dde0d925a04c40919533edae0621779d",
      "e06a621fcc884cf6948861ba70a040d7",
      "e37efdd73e00484e80d22cf19838eb9b",
      "21b154e8d2314b97bac7bf510ce7ee43",
      "05cc15488dee4bfa8294781adc2775e4",
      "3a7c2cd64e754fbe8c8f910f9d76b42d",
      "a0c2d63887744302b6179f36bb583879",
      "277da64d48814482b3f451b8c8706461",
      "f4c9792bf2194999b083495d1379b513",
      "569146c4214f4e4eb7b7de078ebd5857",
      "a03121d1063149a189d3d7f561099d44",
      "d4e80ded4667403fa7cf27dc2706e756",
      "80ec1588af6b426c9d9709989c78d455",
      "57e595d3ab874902a16accd73b19c865",
      "6069b24de11f468f8cbb3360ffd1fb09",
      "5778e23acab7487cbd3c12d70dbbf1ba",
      "e8b1e8864ce245f3bcaa1ac7ae5f54de",
      "4044fc511d4e479782cf80c63ef73e26",
      "bf1fc60432234eb7a005859476802687",
      "a4bfb99a9177451295c08faaf090e309",
      "9ad2750aa5ab4972bc8156a6115dab44",
      "a97bc72ee0c240b2885198db446ca205",
      "e14ae556bac045d5a64f028ce76ded2e",
      "80d44875f8b44dc4b178a105b08bc267",
      "05a250d18cda4fe79f69cb258960faf8",
      "f2a9bc258d06496f8e8e1f2f8ccc515b",
      "4bb1b90b1880493b9bc53341c5988131",
      "67b82af834bf4526b730bfa4fc8d4ad3",
      "133badee9be148eb9daa878d67325bb5",
      "4daedae6cba24c6a835a8e4ae05e322d",
      "9dae0e5082ee4776bc64e644d155af33",
      "f030e21d8d2541c9a1b5d56ecfbbc1bd",
      "74e12fd6e60f4cdaad14590776eac2e8",
      "8246472500284ad18f8a1732bc351bfd",
      "0382e6059946468684c04b7b5c81a7f0",
      "c0c7c2f45c9842728752cb9d45666aa7",
      "2a886f21a01e457b9ea2c8748a3a4f61",
      "5110cc7789b24cff84c4e98c56eb1a56",
      "facfff95c79f44c28ee7fb7743ba15b2",
      "cd6a92e001f7443b912e8bc87ac3721d",
      "63088a4b7fcb497d8db10b9ba6530e20",
      "96e8555d63cf417fa7430110ef1624e5",
      "980f204cee3f4d829c4326fe63d16b0e",
      "4a3fb6b58fb240439161ecc8be792721",
      "a8efeb7e49c8460ca32bcd5c837f8b2b",
      "cabac91dde53470ea0d5e05d8218d04d",
      "060c7285770746819031b482fbda4895",
      "507be369aa374425af36fb96064e610c",
      "a9c1e03886a94994915b30c1146d226e",
      "66e11e6ec71f4a2f87c6219bfa32d526",
      "f8579260247540a1998c25fec1762e2a",
      "d892fd9f6ea84958b4dc80802e07adf7",
      "ca84fa32fa0849dea87342c58d591cce",
      "24d0f4c01b754ca08d8fd47efdd7401d",
      "ef6674bcc2cc493d98836651caf42ae8",
      "7e00183725e742c0b1146b61fd258d49",
      "b21e1f28e1e24c218ee7e5310071a346",
      "ea100e2cb078487a8459b0fa5b21eb9d"
     ]
    },
    "id": "0Gs2m5QtPIp0",
    "outputId": "bcd1d4de-ace4-4973-cfe6-9beb0fe54657"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6b436b11235a41c98bf37edcab77e6a8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)lve/main/config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "21b154e8d2314b97bac7bf510ce7ee43",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6069b24de11f468f8cbb3360ffd1fb09",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)neration_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f2a9bc258d06496f8e8e1f2f8ccc515b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)olve/main/vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2a886f21a01e457b9ea2c8748a3a4f61",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)olve/main/merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "507be369aa374425af36fb96064e610c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "text_generator = pipeline(\"text-generation\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sNIhhyloPIp1",
    "outputId": "b86983a0-5c13-4e80-9203-3929d425f594"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'text-generation': {'do_sample': True, 'max_length': 50}}"
      ]
     },
     "execution_count": 235,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_generator.model.config.task_specific_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {
    "id": "JAlA7qvJPIp1"
   },
   "outputs": [],
   "source": [
    "base_text = \"\"\"\n",
    "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "OGsxW7xVPIp1",
    "outputId": "dbb7c437-72e9-4bba-e731-35e241db5aa7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
      "\n",
      "It turned out that it wasn't really strange that Alice liked books. Her sister would sit down, and listen to the conversation, as if to say `you have some idea,'or `it's the idea of the book and how to read and how it will look when you're done with it, and how to get up, when I have to leave for dinner. But the book was a good thing for her, that she had a good appetite, and she could always catch up; and that while she was doing it, she could be very drunk.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "result = text_generator(base_text, max_length=250)\n",
    "print(result[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Kb1uve8LPIp1"
   },
   "source": [
    "# Putting It All Together"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BBjmamx_PIp1"
   },
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 113,
     "referenced_widgets": [
      "a6176cd161f3434790dc6a57b1dfd434",
      "dac7c2ae49cb4951bf7a85f81659b858",
      "1744cd188d08474884a66b6abc6bdba8",
      "6fac2174a8e64ce0a4b952238892d83d",
      "3eaa5c70d72c42a8bccfeb5c5a352b96",
      "b48d21256d6f4346b6d100b45b129432",
      "2851707e4a4040febfe91a3c11e91b82",
      "aaf53f3e949d40cabae0c0daf2aba48b",
      "91dd08b16fc847b4952ce291dbf40005",
      "3151a4246102474da9da1912025ff995",
      "c57435ffd8de4cea91481bf90804d664",
      "3b3100b518b147849503371651dd4e52",
      "03091f57790c4fdabf03df270c8efdaf",
      "8fa1912665134655ab0908ef74352a16",
      "065124bd0f98460ebac66cd004372b98",
      "6c0631dec58c4ee69369c25e38b045f6",
      "804670cf287b4f80aebdd98a18e017a2",
      "b67b3b33c6c841538d078572154c6e60",
      "81cc5b6d0d5e46f2abab5a40f920609e",
      "76c26fed3e6f4e2fa669e7a4b66a0c52",
      "7394c8e9b7f74285b4d10201797b66f8",
      "7eebc5928db4451d8279994eb108e53e",
      "3cb4349492694a3298d291c8fb1b200b",
      "06dac90ed3454abe9927118db312da01",
      "61e0249981a6424f8a011cc10ef08b88",
      "650796f942bc4b69a9bcd7a868dfd9b6",
      "68ff6fe07e8244cf8eca148e5d595b69",
      "a3531b7aebd54dfd81df3407d90fa2ad",
      "e37ee1e0045d4b66a849426d3db233b6",
      "5ec82e416c714d6f912c1d08afed5349",
      "5be8aae242ce4c79aac0d83f789122a7",
      "73efbdd1defc424dae2b39c40d289785",
      "6b43d9e9587f4cefa6adf350dde8f1f1"
     ]
    },
    "id": "rKf7qtJkPIp1",
    "outputId": "6732ff0a-e6da-4ab6-859c-61b1a3425197"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6176cd161f3434790dc6a57b1dfd434",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3b3100b518b147849503371651dd4e52",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3cb4349492694a3298d291c8fb1b200b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset = load_dataset(path='csv', data_files=['texts/alice28-1476.sent.csv'], quotechar='\\\\', split=Split.TRAIN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {
    "id": "sRxp47A-PIp2"
   },
   "outputs": [],
   "source": [
    "shuffled_dataset = dataset.shuffle(seed=42)\n",
    "split_dataset = shuffled_dataset.train_test_split(test_size=0.2, seed=42)\n",
    "train_dataset, test_dataset = split_dataset['train'], split_dataset['test']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {
    "id": "gVF5IzXsPIp2"
   },
   "outputs": [],
   "source": [
    "auto_tokenizer = AutoTokenizer.from_pretrained('gpt2')\n",
    "def tokenize(row):\n",
    "    return auto_tokenizer(row['sentence'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "a217806554694ed7bf5aa8ad795a3597",
      "5777ee41883d442c91fe94bb09549bc0",
      "f015070e82a44db4a6e54147f59af735",
      "92c12f0acd0b497b8d930e149eed7a7f",
      "28cf349108fb4666b27ae62b4a573b3a",
      "1cf85a231fab45fe8a916c95513992e4",
      "e84b2ead2ebf44a9844271d79b48eaa4",
      "f6954a6ac6274d768555700795ab05dc",
      "5b7ae7c1cecd47c3a9289ef817567975",
      "648574ba3cfd4190ae44289b68d3371f",
      "1a72677b51f745489ce1e6c7083b3c2a",
      "cfb5fd469e194e71b416721c7737bade",
      "5c28a96c917e41bf9b62fedc677d9663",
      "de85477125b640d58d61fe84a1fe410f",
      "ce162b6815a3440f814b88ea0aa283fe",
      "2d0d2338b6c740d9ac55d6514637917c",
      "9a6805866e9a4aaebabfdea1a9754bd8",
      "17fb0f28a2a14c9b91a37a09f21b6e3c",
      "321c38c7fb5a48cabde40783f72fb30e",
      "62da2d8125004d458561df0fdc09d244",
      "7ea378ac749743e2a4d32c5183274d54",
      "8121990bf84a471eaa26d33c4872d835"
     ]
    },
    "id": "YUBOlXsqPIp2",
    "outputId": "6fa4fa65-0216-4a05-c722-ac0c823cc4bd"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a217806554694ed7bf5aa8ad795a3597",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/1289 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cfb5fd469e194e71b416721c7737bade",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/323 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_train_dataset = train_dataset.map(tokenize, remove_columns=['source', 'sentence'], batched=True)\n",
    "tokenized_test_dataset = test_dataset.map(tokenize, remove_columns=['source', 'sentence'], batched=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "m-4cJ4pAPIp2",
    "outputId": "3ba1fade-86be-4730-d43d-9dccb6ff203d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9, 28, 20, 9, 34, 29]"
      ]
     },
     "execution_count": 242,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(map(len, tokenized_train_dataset[0:6]['input_ids']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VKWqqWJ3PIp2"
   },
   "source": [
    "### \"Packed\" Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U7aJwvXsPIp3"
   },
   "source": [
    "![](https://github.com/dvgodoy/PyTorchStepByStep/blob/master/images/block_tokens.png?raw=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "metadata": {
    "id": "7lAMC3OtPIp3"
   },
   "outputs": [],
   "source": [
    "# Adapted from https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_clm.py\n",
    "def group_texts(examples, block_size=128):\n",
    "    # Concatenate all texts.\n",
    "    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}\n",
    "    total_length = len(concatenated_examples[list(examples.keys())[0]])\n",
    "    # We drop the small remainder, we could add padding if the model supported it instead of this drop, you can\n",
    "    # customize this part to your needs.\n",
    "    total_length = (total_length // block_size) * block_size\n",
    "    # Split by chunks of max_len.\n",
    "    result = {\n",
    "        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]\n",
    "        for k, t in concatenated_examples.items()\n",
    "    }\n",
    "    result[\"labels\"] = result[\"input_ids\"].copy()\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "b18c2a607fe049398062c6c207484e3e",
      "f151d215e09a490f910be8db015bba80",
      "382a49f26eb1405d86daa145b4b5786b",
      "a70b5fb152ce46448bdc636835987a24",
      "51d15bcf281e4e03b28a2b4e636f1866",
      "e12ed3b7dc5c4932866f433ef048800e",
      "fb9f5fc23bb447f88cd6ff1f0bb4b35c",
      "ca2f0e916fd94c2788e86ef5aecffe1e",
      "c5bc9c3d584d4a4a83061814de4c37d4",
      "6ff6524cf1454919888e3967c7ca4728",
      "7ae214ca6e9546798b71d7633377b2d2",
      "78131e69fc5e48b98788b7ad919895d8",
      "0bd98fea98fe46c58c031c4afcb65a03",
      "0acdf948027f48668033a6b0dde45a33",
      "fd54ccef45584a80a70e576939c0b345",
      "b37876b0da9543b1bfcfdf01b95205a5",
      "5fde17667ee94504abd7b7bcc3d110ff",
      "3372a0e5e8514968a5bc819235e8616f",
      "4763944fef3f446e880caa9623448b53",
      "1fc592d0c87e440c9d30ff7b985b96f8",
      "47c99e2702e242559b5c247074ee15b7",
      "75131cbbc5024e3498bc919ae56939d7"
     ]
    },
    "id": "TirMA67-PIp3",
    "outputId": "e0a14a88-a6a5-4b98-b3aa-f056e523f7b5"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b18c2a607fe049398062c6c207484e3e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/1289 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "78131e69fc5e48b98788b7ad919895d8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/323 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lm_train_dataset = tokenized_train_dataset.map(group_texts, batched=True)\n",
    "lm_test_dataset = tokenized_test_dataset.map(group_texts, batched=True)\n",
    "lm_train_dataset.set_format(type='torch')\n",
    "lm_test_dataset.set_format(type='torch')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lwOa4nluPIp3",
    "outputId": "a20eae6e-7025-4990-9846-2da319ff89bd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([   63,  2437,   466,   345,   760,   314,  1101,  8805,  8348,   464,\n",
      "         2677,  3114,  7296,  6819,   379,   262,  2635, 25498,    11,   508,\n",
      "          531,   287,   257,  1877,  3809,    11,  4600,  7120, 25788,  1276,\n",
      "         3272,    12,  1069,  9862, 12680,  4973,  2637,  1537,   611,   314,\n",
      "         1101,   407,   262,   976,    11,   262,  1306,  1808,   318,    11,\n",
      "         5338,   287,   262,   995,   716,   314,    30,   464,   360,   579,\n",
      "         1076,  6364,  4721,   465,  2951,    13,    63,  1026,   373,   881,\n",
      "        21289,   272,   353,   379,  1363,  4032,  1807,  3595, 14862,    11,\n",
      "         4600, 12518,   530,  2492,   470,  1464,  3957,  4025,   290,  4833,\n",
      "           11,   290,   852,  6149,   546,   416, 10693,   290, 33043,    13,\n",
      "         1870, 14862,   373,   523,   881, 24776,   326,   673,  4966,   572,\n",
      "          379,  1752,   287,   262,  4571,   340,  6235,   284,    11,  1231,\n",
      "         2111,   284,  4727,   262,  7457,   340,   550,   925])\n"
     ]
    }
   ],
   "source": [
    "print(lm_train_dataset[0]['input_ids'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hkJM5tc3PIp3",
    "outputId": "8fc537b1-f717-42ef-bb1f-90d784ac7286"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(239, 56)"
      ]
     },
     "execution_count": 246,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(lm_train_dataset), len(lm_test_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MgLxoOb-PIp3"
   },
   "source": [
    "## Model Configuration & Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "q9NLtw1APIp4",
    "outputId": "72de724e-e985-407d-d720-8ef3a54210be"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'>\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoModelForCausalLM\n",
    "model = AutoModelForCausalLM.from_pretrained('gpt2')\n",
    "print(model.__class__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4fI5PE_nPIp4",
    "outputId": "8b770d8f-f40d-4021-eec3-d6feed1539be"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Embedding(50257, 768)"
      ]
     },
     "execution_count": 248,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.resize_token_embeddings(len(auto_tokenizer))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {
    "id": "Ta_7BbrVPIp4"
   },
   "outputs": [],
   "source": [
    "training_args = TrainingArguments(\n",
    "    output_dir='output',\n",
    "    num_train_epochs=1,\n",
    "    per_device_train_batch_size=1,\n",
    "    per_device_eval_batch_size=8,\n",
    "    evaluation_strategy='steps',\n",
    "    eval_steps=50,\n",
    "    logging_steps=50,\n",
    "    gradient_accumulation_steps=4,\n",
    "    prediction_loss_only=True,\n",
    ")\n",
    "\n",
    "trainer = Trainer(model=model,\n",
    "                  args=training_args,\n",
    "                  train_dataset=lm_train_dataset,\n",
    "                  eval_dataset=lm_test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3WQN-YhePIp4",
    "outputId": "5c8a8d78-b2c5-4849-f7db-532f271c5d6b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'loss': 3.5634, 'learning_rate': 7.627118644067798e-06, 'epoch': 0.84}\n",
      "{'eval_loss': 3.3356945514678955, 'eval_runtime': 0.6675, 'eval_samples_per_second': 83.899, 'eval_steps_per_second': 10.487, 'epoch': 0.84}\n",
      "{'train_runtime': 16.5379, 'train_samples_per_second': 14.452, 'train_steps_per_second': 3.568, 'train_loss': 3.5582350714732023, 'epoch': 0.99}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=59, training_loss=3.5582350714732023, metrics={'train_runtime': 16.5379, 'train_samples_per_second': 14.452, 'train_steps_per_second': 3.568, 'train_loss': 3.5582350714732023, 'epoch': 0.99})"
      ]
     },
     "execution_count": 250,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_ru9UY-NPIp5",
    "outputId": "4025f499-5971-4e3f-d8da-1ebea3f6342a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'eval_loss': 3.328803777694702, 'eval_runtime': 0.7055, 'eval_samples_per_second': 79.378, 'eval_steps_per_second': 9.922, 'epoch': 0.99}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'eval_loss': 3.328803777694702,\n",
       " 'eval_runtime': 0.7055,\n",
       " 'eval_samples_per_second': 79.378,\n",
       " 'eval_steps_per_second': 9.922,\n",
       " 'epoch': 0.99}"
      ]
     },
     "execution_count": 251,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.evaluate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UYsl0aOiPIp5"
   },
   "source": [
    "## Generating Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {
    "id": "xhkK5TIFPIp5"
   },
   "outputs": [],
   "source": [
    "device_index = model.device.index if model.device.type != 'cpu' else -1\n",
    "gpt2_gen = pipeline('text-generation', model=model, tokenizer=auto_tokenizer, device=device_index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PF0RfSp4PIp5",
    "outputId": "7d2d8f5c-ab5f-4506-b14b-cba022fe65fd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,'thought Alice `without pictures or conversation?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.\n",
      "`You never liked drawing, didn't you?'said the Rabbit, so much so that they almost shook them both out in the air. `Well,'said the Alice again, 'it was as if we had a long and very long table.`There were two more--' The Rabbit was puzzled at the name, and thought its rather strange name had been Alice's favourite, in the last instance to her delight:  `that they should be called the Queen's Cat, because she was, indeed, quite an idiot:        (\n"
     ]
    }
   ],
   "source": [
    "result = gpt2_gen(base_text, max_length=250)\n",
    "print(result[0]['generated_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "DQG_yBYhPIp6"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "014cde879c6b4bf486dbd88bb4f15aba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "017e198210e54a189160e6c5448b45ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "020e13cca93f4c20a5df7384b61b56d8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "028efd79c4aa49f090fde0d01507518e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_77cc41136f654cc289f578ffe99b47f9",
      "placeholder": "​",
      "style": "IPY_MODEL_d229ed8924f44660afd51f751a62074b",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "03091f57790c4fdabf03df270c8efdaf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_804670cf287b4f80aebdd98a18e017a2",
      "placeholder": "​",
      "style": "IPY_MODEL_b67b3b33c6c841538d078572154c6e60",
      "value": "Extracting data files: 100%"
     }
    },
    "0382e6059946468684c04b7b5c81a7f0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "04bd756693f14b21bd2901fc2e73d3ef": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c0ffdf9164164d5c9213c7dd10116f23",
      "max": 629,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4ab5ac2fbe27424db741224634932854",
      "value": 629
     }
    },
    "0591bc04e579467c9ee57bdef9210df5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "05a250d18cda4fe79f69cb258960faf8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "05cc15488dee4bfa8294781adc2775e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f4c9792bf2194999b083495d1379b513",
      "placeholder": "​",
      "style": "IPY_MODEL_569146c4214f4e4eb7b7de078ebd5857",
      "value": "Downloading model.safetensors: 100%"
     }
    },
    "060c7285770746819031b482fbda4895": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "065124bd0f98460ebac66cd004372b98": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7394c8e9b7f74285b4d10201797b66f8",
      "placeholder": "​",
      "style": "IPY_MODEL_7eebc5928db4451d8279994eb108e53e",
      "value": " 1/1 [00:00&lt;00:00, 49.29it/s]"
     }
    },
    "06dac90ed3454abe9927118db312da01": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a3531b7aebd54dfd81df3407d90fa2ad",
      "placeholder": "​",
      "style": "IPY_MODEL_e37ee1e0045d4b66a849426d3db233b6",
      "value": "Generating train split: "
     }
    },
    "078f35a0fcb8401e9e3b39366ccdfa22": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_84904b49bcfb4f70a267c906a4981493",
      "placeholder": "​",
      "style": "IPY_MODEL_931213e9c0ee47a799a98c5a0a57a764",
      "value": " 268M/268M [00:01&lt;00:00, 207MB/s]"
     }
    },
    "089fea1d6a6744ae80454aa101f5898e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5ddaead5423c4750aa5ceabb2f35e860",
       "IPY_MODEL_cb0258347d984bdbac59ec33811f51b5",
       "IPY_MODEL_efd84a7954324854a1d0fc0a533de65e"
      ],
      "layout": "IPY_MODEL_1cd3ac215fd04246af43ad48f3a02633"
     }
    },
    "08d599a00ddb4b348bf3818fd86b098f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "099d406c76b44870be13ee5b3134a7de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "09c09951186c4b8baca1824cece58586": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "09c64d67093745c1aab40091ab7e80d6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0a799174580945b5a21137b528c5da56": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0a85cc9e58ca491cb6ad286fc5afbcbd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e74ffc2c8ab645b99b823dfcd834033e",
       "IPY_MODEL_861d8b0a49d044bba3beff95743a3c47",
       "IPY_MODEL_225b7a1272ad497680eead78e2fb21ba"
      ],
      "layout": "IPY_MODEL_ba91e70666b84e95b4ca4e6e6e1503a5"
     }
    },
    "0acdf948027f48668033a6b0dde45a33": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4763944fef3f446e880caa9623448b53",
      "max": 323,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1fc592d0c87e440c9d30ff7b985b96f8",
      "value": 323
     }
    },
    "0bd98fea98fe46c58c031c4afcb65a03": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5fde17667ee94504abd7b7bcc3d110ff",
      "placeholder": "​",
      "style": "IPY_MODEL_3372a0e5e8514968a5bc819235e8616f",
      "value": "Map: 100%"
     }
    },
    "0bffcdb38d1d4f3c85e93210aabfa404": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0d47571efe944cb3bcd4062deef28a76": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0dcb056b573242769c81edf55126e236": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "110fcccc17cd464fb2b61838c02c45a0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1177ccace23c41cd9d4a64ac9a214106": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11bea8fc0ead466bbb93aec568aaf1ae": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "131e7124d505464f9e62e06ee7344fd9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "133badee9be148eb9daa878d67325bb5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0382e6059946468684c04b7b5c81a7f0",
      "placeholder": "​",
      "style": "IPY_MODEL_c0c7c2f45c9842728752cb9d45666aa7",
      "value": " 1.04M/1.04M [00:00&lt;00:00, 5.72MB/s]"
     }
    },
    "134448f816dd4e1c9fc331e98d7381b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a0d6a4f644164ad5b8886e50f37fdee7",
       "IPY_MODEL_ae459920284243408cf298063ea85720",
       "IPY_MODEL_f6bfeefba2ae41a786d03af7c2875bea"
      ],
      "layout": "IPY_MODEL_ae0ece405c544406b9e0c90a665d068c"
     }
    },
    "14a84284b58c4927a57600bd63b83f60": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_169b92244a9e42cf89ed5d7b6fe643a7",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_89b32c1717c342b28edc5c90be607f0c",
      "value": 3081
     }
    },
    "15a667e533e945c894af026b818db869": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "15d7e137a6ff45a89959fe0794b3894d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "15e6c1e653ea43ceac91c1188d2d9b4b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1628cb8828844f46b2c3d673117e0831": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "162b098de96a4b2084685c0a77a0eacb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e47034499cc34e38bc0caba6d32e0af2",
       "IPY_MODEL_f12561a7885745f68335b541d6e33baa",
       "IPY_MODEL_a229f1dee3d043d49b4f337d21aee795"
      ],
      "layout": "IPY_MODEL_b9f77f25becb47958456c513715fa705"
     }
    },
    "169b92244a9e42cf89ed5d7b6fe643a7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1744cd188d08474884a66b6abc6bdba8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aaf53f3e949d40cabae0c0daf2aba48b",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_91dd08b16fc847b4952ce291dbf40005",
      "value": 1
     }
    },
    "17f9f24135664f2185908a9a686fe7b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "17fb0f28a2a14c9b91a37a09f21b6e3c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "18d184687008423c94560b773dcf41f9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1a6934c3c9614c45a6ab239dcf90bc6a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1a72677b51f745489ce1e6c7083b3c2a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1bcb447d7e7c4325bb5e590a9ad2396a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1cd3ac215fd04246af43ad48f3a02633": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1cf85a231fab45fe8a916c95513992e4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1d110c4fecb74297b1d648948a410fcd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1e3440f0270448baaa0b19928392e7a6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e5982c69228437689fee07d4b252fab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fd480f04a0354a7ab1ac9fd4d531ccc1",
       "IPY_MODEL_805b82526c7c4cb48a1c3dc795b7153f",
       "IPY_MODEL_fa05df95f5b540859234430c7a9f8a92"
      ],
      "layout": "IPY_MODEL_53c46000a715472c813fe4138446826f"
     }
    },
    "1fc592d0c87e440c9d30ff7b985b96f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "1fe2af3e027142309946a9cc019e5c13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "20b0a8dbb55b4458ad16371a5f9617ec": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "21b154e8d2314b97bac7bf510ce7ee43": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_05cc15488dee4bfa8294781adc2775e4",
       "IPY_MODEL_3a7c2cd64e754fbe8c8f910f9d76b42d",
       "IPY_MODEL_a0c2d63887744302b6179f36bb583879"
      ],
      "layout": "IPY_MODEL_277da64d48814482b3f451b8c8706461"
     }
    },
    "225b7a1272ad497680eead78e2fb21ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0591bc04e579467c9ee57bdef9210df5",
      "placeholder": "​",
      "style": "IPY_MODEL_4c76509902e24df2aa811d754d607817",
      "value": " 3852/0 [00:00&lt;00:00, 13691.14 examples/s]"
     }
    },
    "23cf666f7f614867a350e9ef44020040": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "241a564f58564f3690a3955def8e997f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "24b4642b637b40acb9e5787c71989d1a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "24d0f4c01b754ca08d8fd47efdd7401d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "25a7341afc894965bc384be1379e07dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2689aebe8b564cd7b1f174c8b2adcb3e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d2e070514ba74de3ac0ed266d311a1a0",
      "placeholder": "​",
      "style": "IPY_MODEL_684d4d25c7654a24b7e50775fd8f5bc4",
      "value": " 268M/268M [00:01&lt;00:00, 257MB/s]"
     }
    },
    "26e6170d5cd348dfa3a5de35f2e85ec3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "277da64d48814482b3f451b8c8706461": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2851707e4a4040febfe91a3c11e91b82": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "28cf349108fb4666b27ae62b4a573b3a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2973d5f945a74243a90d44c30dd99d6c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "29a95b9254be492ba40595dbd4d56c2b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_099d406c76b44870be13ee5b3134a7de",
      "placeholder": "​",
      "style": "IPY_MODEL_326894b89a9e471d8b3d13eb3c44ebdc",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "2a886f21a01e457b9ea2c8748a3a4f61": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5110cc7789b24cff84c4e98c56eb1a56",
       "IPY_MODEL_facfff95c79f44c28ee7fb7743ba15b2",
       "IPY_MODEL_cd6a92e001f7443b912e8bc87ac3721d"
      ],
      "layout": "IPY_MODEL_63088a4b7fcb497d8db10b9ba6530e20"
     }
    },
    "2b6f2219e25b4f70a21aab7dc67f210d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2bf8da4a7a5341999ef9f95ac5c176c9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5e35c165cbb14ba0a5f3098b362a6ef6",
       "IPY_MODEL_79947dd0ddf34e5192a2678b52533a97",
       "IPY_MODEL_078f35a0fcb8401e9e3b39366ccdfa22"
      ],
      "layout": "IPY_MODEL_671a21b847754ec8b027956e53c5f337"
     }
    },
    "2d0d2338b6c740d9ac55d6514637917c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2db3d91800bd40078fd85d53ea88a042": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2f7c6c45c3354158a526792d8c8324f5",
      "placeholder": "​",
      "style": "IPY_MODEL_aa2e85f8edcc44e3a83828950c0ee1ea",
      "value": " 771/771 [00:00&lt;00:00, 7492.05 examples/s]"
     }
    },
    "2df66cf412d4416aa4d4120ef82f283d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3dcd50ad000b4860bc1fd8f4566d50e4",
      "placeholder": "​",
      "style": "IPY_MODEL_be84a6c67cec43ad81a2defc92698ffd",
      "value": "Downloading model.safetensors: 100%"
     }
    },
    "2e7a9176007c49b8b6b623b3679fcac0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2f7c6c45c3354158a526792d8c8324f5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2f84f62aa77b4738b3eff4bacfe28179": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bebba38c2ec54dcfaaf2097d3dd3f4c3",
      "max": 665,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_dde0d925a04c40919533edae0621779d",
      "value": 665
     }
    },
    "30653f3a5d36426e970064e40ede2418": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_09c64d67093745c1aab40091ab7e80d6",
      "placeholder": "​",
      "style": "IPY_MODEL_511c36e2cdcd4eaeb6db3dd6af3e14c8",
      "value": "Downloading (…)solve/main/vocab.txt: 100%"
     }
    },
    "314323441385434a98cecb6fd4aaf2e3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3151a4246102474da9da1912025ff995": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "321c38c7fb5a48cabde40783f72fb30e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "326894b89a9e471d8b3d13eb3c44ebdc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3372a0e5e8514968a5bc819235e8616f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "344401e1b0234ea394d535dfc2f02da4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "34d97e1f285448d8a67cdd3d3a52ab3c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "36f064f978d34560ad0536674d9115cd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3721e8f28e1b4547aaedcd760bdfd69a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "382a49f26eb1405d86daa145b4b5786b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca2f0e916fd94c2788e86ef5aecffe1e",
      "max": 1289,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c5bc9c3d584d4a4a83061814de4c37d4",
      "value": 1289
     }
    },
    "3991e93f6d5d4f8e8eee2363a34e224c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3a7c2cd64e754fbe8c8f910f9d76b42d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a03121d1063149a189d3d7f561099d44",
      "max": 548105171,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d4e80ded4667403fa7cf27dc2706e756",
      "value": 548105171
     }
    },
    "3b3100b518b147849503371651dd4e52": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_03091f57790c4fdabf03df270c8efdaf",
       "IPY_MODEL_8fa1912665134655ab0908ef74352a16",
       "IPY_MODEL_065124bd0f98460ebac66cd004372b98"
      ],
      "layout": "IPY_MODEL_6c0631dec58c4ee69369c25e38b045f6"
     }
    },
    "3cb4349492694a3298d291c8fb1b200b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_06dac90ed3454abe9927118db312da01",
       "IPY_MODEL_61e0249981a6424f8a011cc10ef08b88",
       "IPY_MODEL_650796f942bc4b69a9bcd7a868dfd9b6"
      ],
      "layout": "IPY_MODEL_68ff6fe07e8244cf8eca148e5d595b69"
     }
    },
    "3d706598c13a404c882ab11ea40ed2ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3dcd50ad000b4860bc1fd8f4566d50e4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3eaa5c70d72c42a8bccfeb5c5a352b96": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3f6619d00b9c4d4ca573b54d6f59d012": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3f81f775fb0c478b8423db9bddbfad2e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3fbaf05579a345c3b2ec0e1fd0698c19": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4044fc511d4e479782cf80c63ef73e26": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_80d44875f8b44dc4b178a105b08bc267",
      "placeholder": "​",
      "style": "IPY_MODEL_05a250d18cda4fe79f69cb258960faf8",
      "value": " 124/124 [00:00&lt;00:00, 6.91kB/s]"
     }
    },
    "4148fdd044b742829876144096e3d247": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b6f29e196ff944b8964a7ccf2a29aae5",
      "placeholder": "​",
      "style": "IPY_MODEL_cbdcba4e4c4c4ec69224ebde008c16a8",
      "value": "Downloading (…)solve/main/vocab.txt: 100%"
     }
    },
    "4341486e91614581909fa67fadc90d48": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "446fed872cbf44e59688454c1fdbebb8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "44b2d5e007b84f85b4f183ba0631500d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e311df68b62b44099c1269883678c94b",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d792c590bc9f4fa18d66294f4c840ce9",
      "value": 231508
     }
    },
    "462794da68e542f381bff271e24185fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e791c7e96e2a4ffdb8f6bdb12e61aa02",
      "placeholder": "​",
      "style": "IPY_MODEL_6756718ae9e14f23858d2d0000d11fce",
      "value": "Downloading (…)solve/main/vocab.txt: 100%"
     }
    },
    "4629c0d2a4ca42d7b347627832900bae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "466c81afbe03406280d6edd92ec0f5aa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "469cb766551f40f992aa5a1522f06796": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bcd028bfbdd44ffbb46c6e140086d03b",
      "placeholder": "​",
      "style": "IPY_MODEL_24b4642b637b40acb9e5787c71989d1a",
      "value": " 1/1 [00:00&lt;00:00, 39.19it/s]"
     }
    },
    "4763944fef3f446e880caa9623448b53": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "47b17d821f924db3a592d734f77ce092": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_15a667e533e945c894af026b818db869",
      "max": 48,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_598a61bb19da461c9257b86a21c963cf",
      "value": 48
     }
    },
    "47c99e2702e242559b5c247074ee15b7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4801fffa2c324ebbac52ed45d6cc5be8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4978f765d5a346ec94880e2a3c1dc73d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "49fbb0bb89e1487fbdcd78e52f9a1d00": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4a3fb6b58fb240439161ecc8be792721": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4ab2a7cacbd4448bb7e1900139ba7f1a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4ab5ac2fbe27424db741224634932854": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4bb1b90b1880493b9bc53341c5988131": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9dae0e5082ee4776bc64e644d155af33",
      "placeholder": "​",
      "style": "IPY_MODEL_f030e21d8d2541c9a1b5d56ecfbbc1bd",
      "value": "Downloading (…)olve/main/vocab.json: 100%"
     }
    },
    "4c76509902e24df2aa811d754d607817": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4d2c3a69a29243088ff2279457f4de62": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4d9eb2d890ea430cb0f6cfb03c0913ec": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1e3440f0270448baaa0b19928392e7a6",
      "max": 3852,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1bcb447d7e7c4325bb5e590a9ad2396a",
      "value": 3852
     }
    },
    "4dacc2dfaec841b9b09643d32cfb142f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f9459b5a73dc44d58a68925f08598e65",
      "placeholder": "​",
      "style": "IPY_MODEL_cb2fea0bc72b4a8486f09d387442624b",
      "value": " 3081/3081 [00:00&lt;00:00, 10158.57 examples/s]"
     }
    },
    "4daedae6cba24c6a835a8e4ae05e322d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4e0e6c72de6b4f01806f07631c00a8e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b63f77ac851147bb8a514d5737135f9a",
       "IPY_MODEL_a000d3139bcc44babd2409da447eab16",
       "IPY_MODEL_8a2aa09b29a243c2a8dca4974d3cbc7b"
      ],
      "layout": "IPY_MODEL_88acd9c1e68642a8b58f9c6d77647d08"
     }
    },
    "4f8ef38fd5614431a7a3c974a7583570": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4feb42875064419fb591079dbd104bfc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_26e6170d5cd348dfa3a5de35f2e85ec3",
      "placeholder": "​",
      "style": "IPY_MODEL_fa9f55afbab04780829b12fe4b116da3",
      "value": "Downloading model.safetensors: 100%"
     }
    },
    "507be369aa374425af36fb96064e610c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a9c1e03886a94994915b30c1146d226e",
       "IPY_MODEL_66e11e6ec71f4a2f87c6219bfa32d526",
       "IPY_MODEL_f8579260247540a1998c25fec1762e2a"
      ],
      "layout": "IPY_MODEL_d892fd9f6ea84958b4dc80802e07adf7"
     }
    },
    "5110cc7789b24cff84c4e98c56eb1a56": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_96e8555d63cf417fa7430110ef1624e5",
      "placeholder": "​",
      "style": "IPY_MODEL_980f204cee3f4d829c4326fe63d16b0e",
      "value": "Downloading (…)olve/main/merges.txt: 100%"
     }
    },
    "511c36e2cdcd4eaeb6db3dd6af3e14c8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "518537d8640346c4bb9544e5011fda3c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8a6ff1cc71c24b37afbdf4e532978c63",
      "placeholder": "​",
      "style": "IPY_MODEL_88ebe60388384796910b1987ca2a7b36",
      "value": " 232k/232k [00:00&lt;00:00, 3.57MB/s]"
     }
    },
    "51d15bcf281e4e03b28a2b4e636f1866": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "52712fe41859413aae6f6b75af710b90": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_462794da68e542f381bff271e24185fa",
       "IPY_MODEL_44b2d5e007b84f85b4f183ba0631500d",
       "IPY_MODEL_7a15f0aba34e4c3e90bfbe605fef9bb5"
      ],
      "layout": "IPY_MODEL_919e99b706034ed6aa81a1e903cfaf73"
     }
    },
    "533e80c06ed342e98f4caf25ec21f03b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "53c46000a715472c813fe4138446826f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "544015661b65451eb7bc40178d156eb7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3fbaf05579a345c3b2ec0e1fd0698c19",
      "placeholder": "​",
      "style": "IPY_MODEL_20b0a8dbb55b4458ad16371a5f9617ec",
      "value": " 48.0/48.0 [00:00&lt;00:00, 2.46kB/s]"
     }
    },
    "569146c4214f4e4eb7b7de078ebd5857": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5716cea9e8d84a289fe0760bcbc76d6a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5777ee41883d442c91fe94bb09549bc0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1cf85a231fab45fe8a916c95513992e4",
      "placeholder": "​",
      "style": "IPY_MODEL_e84b2ead2ebf44a9844271d79b48eaa4",
      "value": "Map: 100%"
     }
    },
    "5778e23acab7487cbd3c12d70dbbf1ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a4bfb99a9177451295c08faaf090e309",
      "placeholder": "​",
      "style": "IPY_MODEL_9ad2750aa5ab4972bc8156a6115dab44",
      "value": "Downloading (…)neration_config.json: 100%"
     }
    },
    "57e595d3ab874902a16accd73b19c865": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "589cc9d8dd0b40d5abb92bb83398f8c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4801fffa2c324ebbac52ed45d6cc5be8",
      "placeholder": "​",
      "style": "IPY_MODEL_97c86ac930a247b49bf111db20690f9f",
      "value": " 28.0/28.0 [00:00&lt;00:00, 1.10kB/s]"
     }
    },
    "58c6a66e017447d990f0383c1fdb57f0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "590434dd89ee48e29e4a8a2c274c5c53": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4ab2a7cacbd4448bb7e1900139ba7f1a",
      "placeholder": "​",
      "style": "IPY_MODEL_4341486e91614581909fa67fadc90d48",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "598a61bb19da461c9257b86a21c963cf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5af30da4716c4bfc96a7ff53ab2255f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d17523ae711f442690b81f29b351d363",
      "placeholder": "​",
      "style": "IPY_MODEL_a03055e17979435892821dcb79ae1524",
      "value": "Map: 100%"
     }
    },
    "5b7ae7c1cecd47c3a9289ef817567975": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5bbcad5e25f74579b2ab8d64e3b77723": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5be8aae242ce4c79aac0d83f789122a7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5c28a96c917e41bf9b62fedc677d9663": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9a6805866e9a4aaebabfdea1a9754bd8",
      "placeholder": "​",
      "style": "IPY_MODEL_17fb0f28a2a14c9b91a37a09f21b6e3c",
      "value": "Map: 100%"
     }
    },
    "5c9d700dd2ec4c0d92231652d9d50758": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5cc744a17a254e739b953518f2b4d47c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5ddaead5423c4750aa5ceabb2f35e860": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_73f44d8a81bf480d831cc2630a054185",
      "placeholder": "​",
      "style": "IPY_MODEL_b778db445dad4a328e2906d989745d53",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "5e35c165cbb14ba0a5f3098b362a6ef6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_670ba37f36524d7699ff3b5be9952b09",
      "placeholder": "​",
      "style": "IPY_MODEL_1a6934c3c9614c45a6ab239dcf90bc6a",
      "value": "Downloading model.safetensors: 100%"
     }
    },
    "5ec82e416c714d6f912c1d08afed5349": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "5fde17667ee94504abd7b7bcc3d110ff": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6069b24de11f468f8cbb3360ffd1fb09": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5778e23acab7487cbd3c12d70dbbf1ba",
       "IPY_MODEL_e8b1e8864ce245f3bcaa1ac7ae5f54de",
       "IPY_MODEL_4044fc511d4e479782cf80c63ef73e26"
      ],
      "layout": "IPY_MODEL_bf1fc60432234eb7a005859476802687"
     }
    },
    "610cb20066c34d948550f9560e498430": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e34df6b7762b42169b6a38cb1db49c79",
      "placeholder": "​",
      "style": "IPY_MODEL_cdad8f4cd3dd4e15be22b26eeb81813d",
      "value": " 232k/232k [00:00&lt;00:00, 3.81MB/s]"
     }
    },
    "61d5d21f585a4c4a8a4e0661ebab0978": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "61e0249981a6424f8a011cc10ef08b88": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5ec82e416c714d6f912c1d08afed5349",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5be8aae242ce4c79aac0d83f789122a7",
      "value": 1
     }
    },
    "61f7bb49e16e486781de91a2975dc83e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "620aa9d515854ad181db286a7fb44ae8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e9905466f9744164971e84d9bbb0354c",
      "placeholder": "​",
      "style": "IPY_MODEL_18d184687008423c94560b773dcf41f9",
      "value": " 771/771 [00:00&lt;00:00, 4497.55 examples/s]"
     }
    },
    "62da2d8125004d458561df0fdc09d244": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "63088a4b7fcb497d8db10b9ba6530e20": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "631233cb0483400eb95bb77b4217f7c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a3d8dcdcb68140cc9a836824d0b301b9",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b00f91de164a49c3bfe1759a685a126f",
      "value": 771
     }
    },
    "632a507543cd43108c278887d0c0e0b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "63a4322a79ce4ed9ade9951a68184c79": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "63cb88869a0e473dacec5a6e43f4731a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_58c6a66e017447d990f0383c1fdb57f0",
      "placeholder": "​",
      "style": "IPY_MODEL_bf91c89808db4dd498cfd7da6dda0cec",
      "value": " 483/483 [00:00&lt;00:00, 37.8kB/s]"
     }
    },
    "648574ba3cfd4190ae44289b68d3371f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "650796f942bc4b69a9bcd7a868dfd9b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_73efbdd1defc424dae2b39c40d289785",
      "placeholder": "​",
      "style": "IPY_MODEL_6b43d9e9587f4cefa6adf350dde8f1f1",
      "value": " 1612/0 [00:00&lt;00:00, 35682.64 examples/s]"
     }
    },
    "66e11e6ec71f4a2f87c6219bfa32d526": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ef6674bcc2cc493d98836651caf42ae8",
      "max": 1355256,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7e00183725e742c0b1146b61fd258d49",
      "value": 1355256
     }
    },
    "670ba37f36524d7699ff3b5be9952b09": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "671a21b847754ec8b027956e53c5f337": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6756718ae9e14f23858d2d0000d11fce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "67b82af834bf4526b730bfa4fc8d4ad3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_74e12fd6e60f4cdaad14590776eac2e8",
      "max": 1042301,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8246472500284ad18f8a1732bc351bfd",
      "value": 1042301
     }
    },
    "68423cc470b84d56b509e43e7d0622fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "684d4d25c7654a24b7e50775fd8f5bc4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "68ff6fe07e8244cf8eca148e5d595b69": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "69782579b2ad412b992f2204b9325dd8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6b436b11235a41c98bf37edcab77e6a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_028efd79c4aa49f090fde0d01507518e",
       "IPY_MODEL_2f84f62aa77b4738b3eff4bacfe28179",
       "IPY_MODEL_c7789d7688fd44159f8680d23616d9ec"
      ],
      "layout": "IPY_MODEL_db7fe2b698aa417fa37b850a5b9ff9ad"
     }
    },
    "6b43d9e9587f4cefa6adf350dde8f1f1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6b5f3c6e7aa941799ea143fbaf6391ed": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6c0631dec58c4ee69369c25e38b045f6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6c7078294d0a48d1b628e396e704497d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6d1dc17b66e54c768c9ccd75f91d3e91": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_826781bbc9f24c7e8cf7d3c7dfb00d01",
       "IPY_MODEL_631233cb0483400eb95bb77b4217f7c4",
       "IPY_MODEL_620aa9d515854ad181db286a7fb44ae8"
      ],
      "layout": "IPY_MODEL_ac23cca0aca54d70bd0c13fca43386d2"
     }
    },
    "6f7870b5ddab4cd6ae6d6df571d624b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6fac2174a8e64ce0a4b952238892d83d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3151a4246102474da9da1912025ff995",
      "placeholder": "​",
      "style": "IPY_MODEL_c57435ffd8de4cea91481bf90804d664",
      "value": " 1/1 [00:00&lt;00:00, 66.90it/s]"
     }
    },
    "6ff4ff5036844be9bbd0993eb6173596": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6ff6524cf1454919888e3967c7ca4728": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "704fabe6625d498ea91de8237e8bc600": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7987211ff4eb48a2a042ccc78a57b964",
      "max": 483,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5bbcad5e25f74579b2ab8d64e3b77723",
      "value": 483
     }
    },
    "70c71e159df441f1819bc5b1ce8be618": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "717a6caa55834da9bdeb736a8bb9192f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_241a564f58564f3690a3955def8e997f",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_11bea8fc0ead466bbb93aec568aaf1ae",
      "value": 231508
     }
    },
    "7394c8e9b7f74285b4d10201797b66f8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "73efbdd1defc424dae2b39c40d289785": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "73f44d8a81bf480d831cc2630a054185": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "74566debe4c24ceabfe4867202dc998f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "74690264747443baada58da696384387": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "74e12fd6e60f4cdaad14590776eac2e8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "75131cbbc5024e3498bc919ae56939d7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "76c26fed3e6f4e2fa669e7a4b66a0c52": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "77cc41136f654cc289f578ffe99b47f9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "78131e69fc5e48b98788b7ad919895d8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_0bd98fea98fe46c58c031c4afcb65a03",
       "IPY_MODEL_0acdf948027f48668033a6b0dde45a33",
       "IPY_MODEL_fd54ccef45584a80a70e576939c0b345"
      ],
      "layout": "IPY_MODEL_b37876b0da9543b1bfcfdf01b95205a5"
     }
    },
    "7987211ff4eb48a2a042ccc78a57b964": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "79947dd0ddf34e5192a2678b52533a97": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0bffcdb38d1d4f3c85e93210aabfa404",
      "max": 267832558,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7ebd8e2c1fcc429896627463d2319bab",
      "value": 267832558
     }
    },
    "7a15f0aba34e4c3e90bfbe605fef9bb5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a000821e9b994a63964db53c2ffae61c",
      "placeholder": "​",
      "style": "IPY_MODEL_c2fb075140864489a7d81c19a0f79e21",
      "value": " 232k/232k [00:00&lt;00:00, 3.83MB/s]"
     }
    },
    "7a9ecdfd864a4fcdb561b580069a7f2e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7ae214ca6e9546798b71d7633377b2d2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7ae512c0087942209b5c83c124bf0eb9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0dcb056b573242769c81edf55126e236",
      "placeholder": "​",
      "style": "IPY_MODEL_2e7a9176007c49b8b6b623b3679fcac0",
      "value": " 3081/3081 [00:00&lt;00:00, 4829.32 examples/s]"
     }
    },
    "7bb936c783ab402b870cf609dd9d3129": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5716cea9e8d84a289fe0760bcbc76d6a",
      "max": 28,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_314323441385434a98cecb6fd4aaf2e3",
      "value": 28
     }
    },
    "7d3433407dec4976b47892803044e899": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7e00183725e742c0b1146b61fd258d49": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7ea378ac749743e2a4d32c5183274d54": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7ebd8e2c1fcc429896627463d2319bab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7eebc5928db4451d8279994eb108e53e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "804670cf287b4f80aebdd98a18e017a2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "805b82526c7c4cb48a1c3dc795b7153f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bff1c8d9ebb64b728c6fc43deff520f9",
      "max": 466062,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_08d599a00ddb4b348bf3818fd86b098f",
      "value": 466062
     }
    },
    "80d44875f8b44dc4b178a105b08bc267": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "80ec1588af6b426c9d9709989c78d455": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8121990bf84a471eaa26d33c4872d835": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "81bb103bdcff45b4955023635434a513": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "81cc5b6d0d5e46f2abab5a40f920609e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8246472500284ad18f8a1732bc351bfd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "826781bbc9f24c7e8cf7d3c7dfb00d01": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a8ca1717ef74473b99d901a76c0028c9",
      "placeholder": "​",
      "style": "IPY_MODEL_1628cb8828844f46b2c3d673117e0831",
      "value": "Map: 100%"
     }
    },
    "84904b49bcfb4f70a267c906a4981493": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "849fd22e48d64438bb066a446c6bebee": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_15d7e137a6ff45a89959fe0794b3894d",
      "placeholder": "​",
      "style": "IPY_MODEL_7a9ecdfd864a4fcdb561b580069a7f2e",
      "value": " 440M/440M [00:05&lt;00:00, 53.6MB/s]"
     }
    },
    "8572ccf616654eca94d5699122aa51ce": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "85f7913cfdb24be095daf7d1610192c6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "861d8b0a49d044bba3beff95743a3c47": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_61d5d21f585a4c4a8a4e0661ebab0978",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9c51062634f544a5afcf04f59dbc9131",
      "value": 1
     }
    },
    "8736176520594608bfe10039ab53d571": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "88acd9c1e68642a8b58f9c6d77647d08": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "88ebe60388384796910b1987ca2a7b36": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "890bcfb8c23046808a22581da6107878": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89856f13c9d247acb31b7802b1cca970": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b143e584661a42969b6a9f376546486a",
       "IPY_MODEL_47b17d821f924db3a592d734f77ce092",
       "IPY_MODEL_544015661b65451eb7bc40178d156eb7"
      ],
      "layout": "IPY_MODEL_d3d5424dca584e38ad28fa4a32fc618b"
     }
    },
    "89b32c1717c342b28edc5c90be607f0c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "89ca84846175417eb38fd7f405a5da93": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8a2aa09b29a243c2a8dca4974d3cbc7b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b3e74b9a6bd64af2905732e61991b1ec",
      "placeholder": "​",
      "style": "IPY_MODEL_df0d7229eeca4861937ea11dcfb855bf",
      "value": " 1/1 [00:00&lt;00:00, 50.03it/s]"
     }
    },
    "8a6ff1cc71c24b37afbdf4e532978c63": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8a72cf42f4e3447987928280478bbcb3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8ac0286de75940438cdaee591a0d3d24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8acc01c4edd249c5bdf36333e9e24e57": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d289b405a544e84a1694b0079a7ce4a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_eaa192d1608f4322854a8bf88242cfa7",
       "IPY_MODEL_e2925c6848c0450b8150dbbc9197f7b4",
       "IPY_MODEL_96787f87f23b4ceaa33955126aa9c252"
      ],
      "layout": "IPY_MODEL_23cf666f7f614867a350e9ef44020040"
     }
    },
    "8fa1912665134655ab0908ef74352a16": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_81cc5b6d0d5e46f2abab5a40f920609e",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_76c26fed3e6f4e2fa669e7a4b66a0c52",
      "value": 1
     }
    },
    "917fb99088eb4329894cef75e45023ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_590434dd89ee48e29e4a8a2c274c5c53",
       "IPY_MODEL_04bd756693f14b21bd2901fc2e73d3ef",
       "IPY_MODEL_94196f53645a4731b78a215090acc5c6"
      ],
      "layout": "IPY_MODEL_69782579b2ad412b992f2204b9325dd8"
     }
    },
    "919e99b706034ed6aa81a1e903cfaf73": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91dd08b16fc847b4952ce291dbf40005": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "92c12f0acd0b497b8d930e149eed7a7f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_648574ba3cfd4190ae44289b68d3371f",
      "placeholder": "​",
      "style": "IPY_MODEL_1a72677b51f745489ce1e6c7083b3c2a",
      "value": " 1289/1289 [00:00&lt;00:00, 2139.08 examples/s]"
     }
    },
    "931213e9c0ee47a799a98c5a0a57a764": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "93fa3f168798430280f1ff68660349a4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_466c81afbe03406280d6edd92ec0f5aa",
      "placeholder": "​",
      "style": "IPY_MODEL_68423cc470b84d56b509e43e7d0622fc",
      "value": " 3852/3852 [00:00&lt;00:00, 18711.23 examples/s]"
     }
    },
    "94196f53645a4731b78a215090acc5c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3991e93f6d5d4f8e8eee2363a34e224c",
      "placeholder": "​",
      "style": "IPY_MODEL_ad73151124ce47fcb81a3c73718296a4",
      "value": " 629/629 [00:00&lt;00:00, 38.0kB/s]"
     }
    },
    "96787f87f23b4ceaa33955126aa9c252": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8572ccf616654eca94d5699122aa51ce",
      "placeholder": "​",
      "style": "IPY_MODEL_61f7bb49e16e486781de91a2975dc83e",
      "value": " 570/570 [00:00&lt;00:00, 26.0kB/s]"
     }
    },
    "96e8555d63cf417fa7430110ef1624e5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "97c86ac930a247b49bf111db20690f9f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "980f204cee3f4d829c4326fe63d16b0e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "980fde34ce5740a09517882962bca862": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b3ad093b5a704c4dbe409b34bbbcb804",
      "max": 440449768,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_632a507543cd43108c278887d0c0e0b3",
      "value": 440449768
     }
    },
    "9a6805866e9a4aaebabfdea1a9754bd8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9ad2750aa5ab4972bc8156a6115dab44": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9c51062634f544a5afcf04f59dbc9131": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9dae0e5082ee4776bc64e644d155af33": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9dbde3e27b4c4440b7a6de971c838ddc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ff1933d7656e41cfa276d74b0d458a5d",
       "IPY_MODEL_4d9eb2d890ea430cb0f6cfb03c0913ec",
       "IPY_MODEL_93fa3f168798430280f1ff68660349a4"
      ],
      "layout": "IPY_MODEL_63a4322a79ce4ed9ade9951a68184c79"
     }
    },
    "a000821e9b994a63964db53c2ffae61c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a000d3139bcc44babd2409da447eab16": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_74690264747443baada58da696384387",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_1fe2af3e027142309946a9cc019e5c13",
      "value": 1
     }
    },
    "a03055e17979435892821dcb79ae1524": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a03121d1063149a189d3d7f561099d44": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a07ca3c194074fc382bce90671da91dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5cc744a17a254e739b953518f2b4d47c",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f59beaa8010b4e4fa173ae0e4bb0ebbd",
      "value": 3081
     }
    },
    "a0c2d63887744302b6179f36bb583879": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_80ec1588af6b426c9d9709989c78d455",
      "placeholder": "​",
      "style": "IPY_MODEL_57e595d3ab874902a16accd73b19c865",
      "value": " 548M/548M [00:07&lt;00:00, 33.2MB/s]"
     }
    },
    "a0d6a4f644164ad5b8886e50f37fdee7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aed9c03c8e9d4e0f8eda236bd013f637",
      "placeholder": "​",
      "style": "IPY_MODEL_e01d1f7a9092482f8b03bee5770116bb",
      "value": "Map: 100%"
     }
    },
    "a19cf0f8087a4b77b6f63e687f4c66eb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2df66cf412d4416aa4d4120ef82f283d",
       "IPY_MODEL_d3b257e5f68f493b9426b9169f8201be",
       "IPY_MODEL_2689aebe8b564cd7b1f174c8b2adcb3e"
      ],
      "layout": "IPY_MODEL_3f81f775fb0c478b8423db9bddbfad2e"
     }
    },
    "a1f68db900cd4f86a7add2b3866ce6e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4148fdd044b742829876144096e3d247",
       "IPY_MODEL_717a6caa55834da9bdeb736a8bb9192f",
       "IPY_MODEL_610cb20066c34d948550f9560e498430"
      ],
      "layout": "IPY_MODEL_3721e8f28e1b4547aaedcd760bdfd69a"
     }
    },
    "a217806554694ed7bf5aa8ad795a3597": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5777ee41883d442c91fe94bb09549bc0",
       "IPY_MODEL_f015070e82a44db4a6e54147f59af735",
       "IPY_MODEL_92c12f0acd0b497b8d930e149eed7a7f"
      ],
      "layout": "IPY_MODEL_28cf349108fb4666b27ae62b4a573b3a"
     }
    },
    "a229f1dee3d043d49b4f337d21aee795": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e413a8f688144c74ba4028bf4ed854d5",
      "placeholder": "​",
      "style": "IPY_MODEL_1177ccace23c41cd9d4a64ac9a214106",
      "value": " 771/771 [00:12&lt;00:00, 76.97 examples/s]"
     }
    },
    "a3531b7aebd54dfd81df3407d90fa2ad": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a3d8dcdcb68140cc9a836824d0b301b9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a42616c094cb4485920db4bb2972b8e8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a4bfb99a9177451295c08faaf090e309": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a6176cd161f3434790dc6a57b1dfd434": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_dac7c2ae49cb4951bf7a85f81659b858",
       "IPY_MODEL_1744cd188d08474884a66b6abc6bdba8",
       "IPY_MODEL_6fac2174a8e64ce0a4b952238892d83d"
      ],
      "layout": "IPY_MODEL_3eaa5c70d72c42a8bccfeb5c5a352b96"
     }
    },
    "a70b5fb152ce46448bdc636835987a24": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6ff6524cf1454919888e3967c7ca4728",
      "placeholder": "​",
      "style": "IPY_MODEL_7ae214ca6e9546798b71d7633377b2d2",
      "value": " 1289/1289 [00:00&lt;00:00, 4697.54 examples/s]"
     }
    },
    "a8ca1717ef74473b99d901a76c0028c9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a8efeb7e49c8460ca32bcd5c837f8b2b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a8f94a7dcd194cee8c44f683dd80b9d8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a97bc72ee0c240b2885198db446ca205": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a9c1e03886a94994915b30c1146d226e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ca84fa32fa0849dea87342c58d591cce",
      "placeholder": "​",
      "style": "IPY_MODEL_24d0f4c01b754ca08d8fd47efdd7401d",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "aa2e85f8edcc44e3a83828950c0ee1ea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "aaf53f3e949d40cabae0c0daf2aba48b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ac23cca0aca54d70bd0c13fca43386d2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ad73151124ce47fcb81a3c73718296a4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ae0ece405c544406b9e0c90a665d068c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ae459920284243408cf298063ea85720": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_89ca84846175417eb38fd7f405a5da93",
      "max": 3081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_25a7341afc894965bc384be1379e07dd",
      "value": 3081
     }
    },
    "aed9c03c8e9d4e0f8eda236bd013f637": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "afb9bfef678240eea5b8890258302834": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_29a95b9254be492ba40595dbd4d56c2b",
       "IPY_MODEL_7bb936c783ab402b870cf609dd9d3129",
       "IPY_MODEL_589cc9d8dd0b40d5abb92bb83398f8c6"
      ],
      "layout": "IPY_MODEL_6b5f3c6e7aa941799ea143fbaf6391ed"
     }
    },
    "b00f91de164a49c3bfe1759a685a126f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "b143e584661a42969b6a9f376546486a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8ac0286de75940438cdaee591a0d3d24",
      "placeholder": "​",
      "style": "IPY_MODEL_6c7078294d0a48d1b628e396e704497d",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "b18c2a607fe049398062c6c207484e3e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f151d215e09a490f910be8db015bba80",
       "IPY_MODEL_382a49f26eb1405d86daa145b4b5786b",
       "IPY_MODEL_a70b5fb152ce46448bdc636835987a24"
      ],
      "layout": "IPY_MODEL_51d15bcf281e4e03b28a2b4e636f1866"
     }
    },
    "b1b0bd4a30364fb79620dc284fb591fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_30653f3a5d36426e970064e40ede2418",
       "IPY_MODEL_f37ddc958e61436795033581348ee8ff",
       "IPY_MODEL_518537d8640346c4bb9544e5011fda3c"
      ],
      "layout": "IPY_MODEL_e93a503d473b411dbf651736f2697704"
     }
    },
    "b21e1f28e1e24c218ee7e5310071a346": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b37876b0da9543b1bfcfdf01b95205a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b3ad093b5a704c4dbe409b34bbbcb804": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b3e74b9a6bd64af2905732e61991b1ec": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b48d21256d6f4346b6d100b45b129432": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b63f77ac851147bb8a514d5737135f9a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2973d5f945a74243a90d44c30dd99d6c",
      "placeholder": "​",
      "style": "IPY_MODEL_17f9f24135664f2185908a9a686fe7b8",
      "value": "Downloading data files: 100%"
     }
    },
    "b67b3b33c6c841538d078572154c6e60": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b6cd1e7ed31d468baa7e9a322ea4b36a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b6f29e196ff944b8964a7ccf2a29aae5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b778db445dad4a328e2906d989745d53": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b7f1693ebfac4297b33ff1e4f7091ff7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4feb42875064419fb591079dbd104bfc",
       "IPY_MODEL_980fde34ce5740a09517882962bca862",
       "IPY_MODEL_849fd22e48d64438bb066a446c6bebee"
      ],
      "layout": "IPY_MODEL_34d97e1f285448d8a67cdd3d3a52ab3c"
     }
    },
    "b8207f4f24304030b4d5944afa13137d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d22fe83a00414a7db643fc242c0767ac",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_70c71e159df441f1819bc5b1ce8be618",
      "value": 771
     }
    },
    "b9f77f25becb47958456c513715fa705": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ba91e70666b84e95b4ca4e6e6e1503a5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bafc4fafea2344f4a7d2def7da6153d2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bc52309682db4f1daac4df480b481dd4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ef6585ad9f58426486fe68e416a5085b",
       "IPY_MODEL_a07ca3c194074fc382bce90671da91dc",
       "IPY_MODEL_4dacc2dfaec841b9b09643d32cfb142f"
      ],
      "layout": "IPY_MODEL_014cde879c6b4bf486dbd88bb4f15aba"
     }
    },
    "bcd028bfbdd44ffbb46c6e140086d03b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "be84a6c67cec43ad81a2defc92698ffd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bebba38c2ec54dcfaaf2097d3dd3f4c3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf1fc60432234eb7a005859476802687": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf91c89808db4dd498cfd7da6dda0cec": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "bff1c8d9ebb64b728c6fc43deff520f9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c0c7c2f45c9842728752cb9d45666aa7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c0ffdf9164164d5c9213c7dd10116f23": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c2fb075140864489a7d81c19a0f79e21": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c57435ffd8de4cea91481bf90804d664": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c5bc9c3d584d4a4a83061814de4c37d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c5fe4d01057141068e6bb47ace62576d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e7d783b2428a4ee487dc4e700e4ed9fc",
       "IPY_MODEL_704fabe6625d498ea91de8237e8bc600",
       "IPY_MODEL_63cb88869a0e473dacec5a6e43f4731a"
      ],
      "layout": "IPY_MODEL_e9cd1445c89746eeacc43611394f05e6"
     }
    },
    "c68e3cfe691a4e17bff5dd0d6fa645d3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_deea6050951d4ca58e70d9a690dbbed9",
       "IPY_MODEL_f2cbe349777d4fb28e1d02ade5f6f63a",
       "IPY_MODEL_d5224726057743ee8d441a0827240ea8"
      ],
      "layout": "IPY_MODEL_85f7913cfdb24be095daf7d1610192c6"
     }
    },
    "c7789d7688fd44159f8680d23616d9ec": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e06a621fcc884cf6948861ba70a040d7",
      "placeholder": "​",
      "style": "IPY_MODEL_e37efdd73e00484e80d22cf19838eb9b",
      "value": " 665/665 [00:00&lt;00:00, 9.32kB/s]"
     }
    },
    "c920c622ebca4d4484ae524879220748": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5af30da4716c4bfc96a7ff53ab2255f1",
       "IPY_MODEL_b8207f4f24304030b4d5944afa13137d",
       "IPY_MODEL_2db3d91800bd40078fd85d53ea88a042"
      ],
      "layout": "IPY_MODEL_a8f94a7dcd194cee8c44f683dd80b9d8"
     }
    },
    "ca2f0e916fd94c2788e86ef5aecffe1e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ca84fa32fa0849dea87342c58d591cce": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cabac91dde53470ea0d5e05d8218d04d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cb0258347d984bdbac59ec33811f51b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_131e7124d505464f9e62e06ee7344fd9",
      "max": 466062,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4d2c3a69a29243088ff2279457f4de62",
      "value": 466062
     }
    },
    "cb2fea0bc72b4a8486f09d387442624b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cbdcba4e4c4c4ec69224ebde008c16a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cd6a92e001f7443b912e8bc87ac3721d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cabac91dde53470ea0d5e05d8218d04d",
      "placeholder": "​",
      "style": "IPY_MODEL_060c7285770746819031b482fbda4895",
      "value": " 456k/456k [00:00&lt;00:00, 7.50MB/s]"
     }
    },
    "cdad8f4cd3dd4e15be22b26eeb81813d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ce162b6815a3440f814b88ea0aa283fe": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7ea378ac749743e2a4d32c5183274d54",
      "placeholder": "​",
      "style": "IPY_MODEL_8121990bf84a471eaa26d33c4872d835",
      "value": " 323/323 [00:00&lt;00:00, 860.51 examples/s]"
     }
    },
    "cfb5fd469e194e71b416721c7737bade": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5c28a96c917e41bf9b62fedc677d9663",
       "IPY_MODEL_de85477125b640d58d61fe84a1fe410f",
       "IPY_MODEL_ce162b6815a3440f814b88ea0aa283fe"
      ],
      "layout": "IPY_MODEL_2d0d2338b6c740d9ac55d6514637917c"
     }
    },
    "d0f6ba92d7484d9aa9ddc84f5318ccba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d17523ae711f442690b81f29b351d363": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d229ed8924f44660afd51f751a62074b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d22fe83a00414a7db643fc242c0767ac": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d2d26ca83c384e8cb8cf9996e1202c76": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4629c0d2a4ca42d7b347627832900bae",
      "placeholder": "​",
      "style": "IPY_MODEL_49fbb0bb89e1487fbdcd78e52f9a1d00",
      "value": "Extracting data files: 100%"
     }
    },
    "d2e070514ba74de3ac0ed266d311a1a0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3b257e5f68f493b9426b9169f8201be": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_446fed872cbf44e59688454c1fdbebb8",
      "max": 267954768,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e3147854e3e849cd8badb22bca577f83",
      "value": 267954768
     }
    },
    "d3d5424dca584e38ad28fa4a32fc618b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d4e80ded4667403fa7cf27dc2706e756": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d5224726057743ee8d441a0827240ea8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_533e80c06ed342e98f4caf25ec21f03b",
      "placeholder": "​",
      "style": "IPY_MODEL_4978f765d5a346ec94880e2a3c1dc73d",
      "value": " 28.0/28.0 [00:00&lt;00:00, 2.01kB/s]"
     }
    },
    "d792c590bc9f4fa18d66294f4c840ce9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d892fd9f6ea84958b4dc80802e07adf7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dac7c2ae49cb4951bf7a85f81659b858": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b48d21256d6f4346b6d100b45b129432",
      "placeholder": "​",
      "style": "IPY_MODEL_2851707e4a4040febfe91a3c11e91b82",
      "value": "Downloading data files: 100%"
     }
    },
    "db7fe2b698aa417fa37b850a5b9ff9ad": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dc2d8ab4dab447a5bfca5f7df1db3b4f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dc64bd6c586c44649346778761ee47dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_d2d26ca83c384e8cb8cf9996e1202c76",
       "IPY_MODEL_e90b1dcef213438a91676618c66d5f85",
       "IPY_MODEL_469cb766551f40f992aa5a1522f06796"
      ],
      "layout": "IPY_MODEL_7d3433407dec4976b47892803044e899"
     }
    },
    "dde0d925a04c40919533edae0621779d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "de85477125b640d58d61fe84a1fe410f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_321c38c7fb5a48cabde40783f72fb30e",
      "max": 323,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_62da2d8125004d458561df0fdc09d244",
      "value": 323
     }
    },
    "deea6050951d4ca58e70d9a690dbbed9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a42616c094cb4485920db4bb2972b8e8",
      "placeholder": "​",
      "style": "IPY_MODEL_b6cd1e7ed31d468baa7e9a322ea4b36a",
      "value": "Downloading (…)okenizer_config.json: 100%"
     }
    },
    "df0d7229eeca4861937ea11dcfb855bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e01d1f7a9092482f8b03bee5770116bb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e06a621fcc884cf6948861ba70a040d7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e12ed3b7dc5c4932866f433ef048800e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e14ae556bac045d5a64f028ce76ded2e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e2925c6848c0450b8150dbbc9197f7b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2b6f2219e25b4f70a21aab7dc67f210d",
      "max": 570,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0d47571efe944cb3bcd4062deef28a76",
      "value": 570
     }
    },
    "e311df68b62b44099c1269883678c94b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e3147854e3e849cd8badb22bca577f83": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e34df6b7762b42169b6a38cb1db49c79": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e37ee1e0045d4b66a849426d3db233b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e37efdd73e00484e80d22cf19838eb9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e413a8f688144c74ba4028bf4ed854d5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e47034499cc34e38bc0caba6d32e0af2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_15e6c1e653ea43ceac91c1188d2d9b4b",
      "placeholder": "​",
      "style": "IPY_MODEL_1d110c4fecb74297b1d648948a410fcd",
      "value": "Map: 100%"
     }
    },
    "e5063a3999bd4629ab6ab8ce92dc647f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e6fc878e9bd94c44a1121ed9fb94af18": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e74ffc2c8ab645b99b823dfcd834033e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f56178de4e344ada9c275e63262ccc77",
      "placeholder": "​",
      "style": "IPY_MODEL_8a72cf42f4e3447987928280478bbcb3",
      "value": "Generating train split: "
     }
    },
    "e7803472ac394d368b137951385998ed": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e791c7e96e2a4ffdb8f6bdb12e61aa02": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e7d783b2428a4ee487dc4e700e4ed9fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_09c09951186c4b8baca1824cece58586",
      "placeholder": "​",
      "style": "IPY_MODEL_6f7870b5ddab4cd6ae6d6df571d624b0",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "e84b2ead2ebf44a9844271d79b48eaa4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e8b1e8864ce245f3bcaa1ac7ae5f54de": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a97bc72ee0c240b2885198db446ca205",
      "max": 124,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e14ae556bac045d5a64f028ce76ded2e",
      "value": 124
     }
    },
    "e90b1dcef213438a91676618c66d5f85": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bafc4fafea2344f4a7d2def7da6153d2",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fd192c24c0e44da0b3cf39fb1fd60b5f",
      "value": 1
     }
    },
    "e93a503d473b411dbf651736f2697704": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e9905466f9744164971e84d9bbb0354c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e9cd1445c89746eeacc43611394f05e6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea100e2cb078487a8459b0fa5b21eb9d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eaa192d1608f4322854a8bf88242cfa7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_890bcfb8c23046808a22581da6107878",
      "placeholder": "​",
      "style": "IPY_MODEL_110fcccc17cd464fb2b61838c02c45a0",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "ef6585ad9f58426486fe68e416a5085b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_36f064f978d34560ad0536674d9115cd",
      "placeholder": "​",
      "style": "IPY_MODEL_74566debe4c24ceabfe4867202dc998f",
      "value": "Map: 100%"
     }
    },
    "ef6674bcc2cc493d98836651caf42ae8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "efd84a7954324854a1d0fc0a533de65e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8736176520594608bfe10039ab53d571",
      "placeholder": "​",
      "style": "IPY_MODEL_f898adb2fc6046329e9abae53870ce29",
      "value": " 466k/466k [00:00&lt;00:00, 3.59MB/s]"
     }
    },
    "f015070e82a44db4a6e54147f59af735": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f6954a6ac6274d768555700795ab05dc",
      "max": 1289,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5b7ae7c1cecd47c3a9289ef817567975",
      "value": 1289
     }
    },
    "f030e21d8d2541c9a1b5d56ecfbbc1bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f12561a7885745f68335b541d6e33baa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e6fc878e9bd94c44a1121ed9fb94af18",
      "max": 771,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_81bb103bdcff45b4955023635434a513",
      "value": 771
     }
    },
    "f1315b0496244e24addeb478d9e64eb8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fd4997c1fe6743218f8bfcf5cfc8f1bd",
       "IPY_MODEL_14a84284b58c4927a57600bd63b83f60",
       "IPY_MODEL_7ae512c0087942209b5c83c124bf0eb9"
      ],
      "layout": "IPY_MODEL_e5063a3999bd4629ab6ab8ce92dc647f"
     }
    },
    "f151d215e09a490f910be8db015bba80": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e12ed3b7dc5c4932866f433ef048800e",
      "placeholder": "​",
      "style": "IPY_MODEL_fb9f5fc23bb447f88cd6ff1f0bb4b35c",
      "value": "Map: 100%"
     }
    },
    "f2a9bc258d06496f8e8e1f2f8ccc515b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4bb1b90b1880493b9bc53341c5988131",
       "IPY_MODEL_67b82af834bf4526b730bfa4fc8d4ad3",
       "IPY_MODEL_133badee9be148eb9daa878d67325bb5"
      ],
      "layout": "IPY_MODEL_4daedae6cba24c6a835a8e4ae05e322d"
     }
    },
    "f2cbe349777d4fb28e1d02ade5f6f63a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_344401e1b0234ea394d535dfc2f02da4",
      "max": 28,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_017e198210e54a189160e6c5448b45ff",
      "value": 28
     }
    },
    "f37ddc958e61436795033581348ee8ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_020e13cca93f4c20a5df7384b61b56d8",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5c9d700dd2ec4c0d92231652d9d50758",
      "value": 231508
     }
    },
    "f4c9792bf2194999b083495d1379b513": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f56178de4e344ada9c275e63262ccc77": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f59beaa8010b4e4fa173ae0e4bb0ebbd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f6954a6ac6274d768555700795ab05dc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f6bfeefba2ae41a786d03af7c2875bea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0a799174580945b5a21137b528c5da56",
      "placeholder": "​",
      "style": "IPY_MODEL_d0f6ba92d7484d9aa9ddc84f5318ccba",
      "value": " 3081/3081 [00:50&lt;00:00, 77.62 examples/s]"
     }
    },
    "f8579260247540a1998c25fec1762e2a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b21e1f28e1e24c218ee7e5310071a346",
      "placeholder": "​",
      "style": "IPY_MODEL_ea100e2cb078487a8459b0fa5b21eb9d",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 5.25MB/s]"
     }
    },
    "f898adb2fc6046329e9abae53870ce29": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f9459b5a73dc44d58a68925f08598e65": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fa05df95f5b540859234430c7a9f8a92": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fcbc067f13494ff2b9798686ab0a8547",
      "placeholder": "​",
      "style": "IPY_MODEL_dc2d8ab4dab447a5bfca5f7df1db3b4f",
      "value": " 466k/466k [00:00&lt;00:00, 6.78MB/s]"
     }
    },
    "fa9f55afbab04780829b12fe4b116da3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "facfff95c79f44c28ee7fb7743ba15b2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4a3fb6b58fb240439161ecc8be792721",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_a8efeb7e49c8460ca32bcd5c837f8b2b",
      "value": 456318
     }
    },
    "fb9f5fc23bb447f88cd6ff1f0bb4b35c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fcbc067f13494ff2b9798686ab0a8547": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fd192c24c0e44da0b3cf39fb1fd60b5f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "fd480f04a0354a7ab1ac9fd4d531ccc1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e7803472ac394d368b137951385998ed",
      "placeholder": "​",
      "style": "IPY_MODEL_6ff4ff5036844be9bbd0993eb6173596",
      "value": "Downloading (…)/main/tokenizer.json: 100%"
     }
    },
    "fd4997c1fe6743218f8bfcf5cfc8f1bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4f8ef38fd5614431a7a3c974a7583570",
      "placeholder": "​",
      "style": "IPY_MODEL_3f6619d00b9c4d4ca573b54d6f59d012",
      "value": "Map: 100%"
     }
    },
    "fd54ccef45584a80a70e576939c0b345": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_47c99e2702e242559b5c247074ee15b7",
      "placeholder": "​",
      "style": "IPY_MODEL_75131cbbc5024e3498bc919ae56939d7",
      "value": " 323/323 [00:00&lt;00:00, 3950.08 examples/s]"
     }
    },
    "ff1933d7656e41cfa276d74b0d458a5d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8acc01c4edd249c5bdf36333e9e24e57",
      "placeholder": "​",
      "style": "IPY_MODEL_3d706598c13a404c882ab11ea40ed2ba",
      "value": "Map: 100%"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
